Jonathon Karelse: ”Why do we forecast the way we do?”

From the Oracle of Delphi right through to Box/Jenkins and from exponential smoothing to S&OP, no stone of business forecasting history is left unturned in the book called Histories of the Future, published by the Canadian management consultant and supply chain specialist Jonathon Karelse. “I wanted to investigate why we actually forecast the way we do.” He made some extraordinary discoveries in the process, as he explains in this interview.

By Harm Beerens

Why do we forecast the way we do? That is the key question Jonathon Karelse sets out to answer in his book. He has traced the entire history of predicting future business performance and listed all the milestones. This has led to an impressive and often very revealing book. For example, artificial intelligence (AI) – despite its current popularity – comes off badly. Karelse soberly observes that the basic principles were actually devised in World War II (by the legendary Alan Turing) and that AI has almost always led to disappointing results in practice. Another eye-opener is that the manual ‘enrichment’ of statistical forecasts by Sales, which happens in virtually every company, on average leads to a 7% less reliable forecast.

Besides these interesting observations, there are many other reasons to read this book. Jonathon Karelse makes us realize that although we are nowadays somewhat scornful of earlier forecasters, such as the Oracle of Delphi or American astrologers who predicted stock prices in the 1920s, in their own times they were taken very seriously by everyone. So Karelse’s underlying message is how do we know that, 20 years from now, people won’t be just as scornful of our current forecasting methods? We have a high opinion of ourselves, with our computer technology and our knowledge of econometrics, yet we can still be caught just as unawares by a financial crisis as Americans were by the Great Crash in 1929.

What is the most important milestone in the history of business forecasting?

“I describe several milestones in my book, but I think the major breakthrough was the maturing of statistical forecasting techniques, followed by the emergence of PCs and the wide availability of powerful computers. This allowed the major ERP vendors to integrate forecasting modules into their standard software in the 1990s, allowing virtually all companies to benefit from them. At its core, forecasting software has not changed much since that time. Software vendors who claim otherwise and say they offer cutting-edge forecasting functionality are really talking nonsense; they are all still using techniques that were developed back in the 1950s and 1960s,” says Jonathon Karelse.

”But forecasting software vendors can still set themselves apart through the usability of their software and how they support the human factor and collaborative forecasting. In fact, this is actually much more important. I sometimes see companies spending an excessive amount of energy on selecting the right forecasting package, when that’s really not what makes the difference. Many companies tend to forget that people and processes are equally important tools for successful supply chain performance.”

You don’t seem to be very impressed by artificial intelligence. How come?

“There’s a lot of hype around artificial intelligence right now, but the underlying concept was actually developed by Alan Turing before World War II. When you ask today’s managers what kind of forecasting system they want, they all say ‘It doesn’t matter, as long as it’s AI-enabled!’, but when you ask them how exactly it should be AI-enabled, they remain silent. Don’t get me wrong… the enormous computing power of today’s computers and the increasing availability of data means that we can now forecast much better and faster than before. And there are certainly examples of where AI and machine learning are achieving much higher forecast reliability than traditional, statistical forecasting techniques. But let’s not suddenly view AI as the autopilot system that means we don’t need anything else,” Jonathon Karelse says.

”What I particularly struggle with is the word ‘intelligence’. Marketing-wise it sounds very nice, of course, but what is now being sold as artificial intelligence is actually just process automation: systems that can process a lot of data very quickly. In my experience, intelligence is something different. Intelligence means being able to reason logically, develop concepts and truly understand customer behaviour. For that, you really need people – demand planners who understand both how markets work and the underlying forecasting techniques. The big disadvantage of the hype surrounding AI is that its results will inevitably be disappointing in supply chain management. AI is great for robotics and companies like Uber and Google, but I don’t think there will be many companies in the future that demonstrably improve their supply chain performance thanks to AI.”

In your book, you talk a lot about behavioural economics. Why is that so important?

“Biases have a much bigger impact on forecasts than we realize. People are subjective creatures and do not act rationally. Anyone who has ever played the Beer Game knows that people tend to overreact to recent data. In a forecasting process, sales figures are invariably rounded up rather than down. If data does not conform to our preconceptions, we dismiss the incidents as outliers or irrelevant. We’ve known all this for a long time but, strangely enough, we’re not really doing anything about it. In a forecasting process, users are extensively empowered to make manual adjustments, when really we should be protecting them from themselves, such as by proactively showing them how their previous manual interventions had disastrous impact.”

Even during sales & operations planning (S&OP) meetings, behavioural economics plays a big role”, states Karelse. ”Of course it’s very good that departments come together to jointly solve their strategic problems, but remember that during such a meeting everyone’s reasoning is always based on their own department’s interests. In fact, that’s actively encouraged. We want our salespeople to remain enthusiastic rather than being discouraged by potential risks, don’t we? And we want an operations manager to cut costs, spend as little as possible and reduce inventories. If you’re unaware of the biases you create by doing this, you will never have an optimal S&OP process. There are several ways to avoid this. One is to point people’s own biases out to them. If I show you that you are biased and that you invariably present things in a more positive light than they deserve, you will probably take that into account in the future. That’s already an important step forward.”

Lastly, what is your key message for our readers?

“Take a critical look at how forecasting is done within your company. Ask why it is done that way, and distrust anyone who says ‘Just because’. Why is a certain algorithm being used? What are the assumptions based on? That’s why I started writing this book. A long time ago, the very first time that I had to do a forecast for the company I was working for back then, I was told exactly how to do it: these are our key customers, this is how they behave, this product will only be sold in this season, and so on. Yet I saw that we had low forecast reliability and that customers often behaved very differently. When I started looking into it, those so-called facts turned out not to be facts at all, but derived from some kind of black box.”

And even when forecasting techniques are used, they are sometimes based on completely incorrect assumptions, states the author. ”One good example is the so-called optimal alpha value in exponential smoothing, a forecasting technique that was first used in 1940 and further refined in the 1950s by Robert Goodell Brown. The alpha value describes the weighting of older data in proportion to recent data. Because Brown himself used the value 0.1 for this, everyone assumed that this value was the most optimal; it became a sort of rule of thumb. Later it turned out that Brown had chosen this number for a very different reason. There were no computers in those days, and so he had set the alpha value at 0.1 purely for convenience, to simplify the calculations. It meant he didn’t have to think so hard and could simply move the decimal point one place across every time… So my message is, always apply critical thinking when forecasting and don’t just blindly trust the history.”

On 16 November, Supply Chain Media is organizing the S&OP Selection Day in the Dutch city of Utrecht. Jonathon Karelse will be the keynote speaker during this international event focused on the selection and implementation of S&OP software. Karelse is regarded as an authority on the topic of forecasting, predictive analytics and S&OP. He is a consultant, publicist and CEO of the consulting firm Northfind Management.