Benchmarking Forecast Accuracy

Forecasting off pipeline stages is like using self-tanner before a beach trip. It gives you false confidence, washes off fast, and fools absolutely no one who gets too close. “30 opps in stage 3 × 40% = $1.2M forecasted.” Bueno. Now subtract the 9 deals that haven’t moved in 30+ days. Then subtract the 5 with no economic buyer involved. And the 8 that don’t have next steps or a MAP. Still $1.2M? lol nah...didn't think so. Stage-based forecasting is pretty broken, mainly because pipeline stages are opinions. Velocity and conversion, on the other hand, are facts. Buyers don’t care what CRM column they’re sitting in. They care about friction, fit, and fear. And your forecast should reflect all three. Here’s how to fix it: 1. Pair conversion rate with conversion velocity. - Let’s say Stage 3 deals have a 30% win rate.  - But they take 52 days to close on average. - If it’s day 50 of the quarter, and that deal just hit Stage 3? It’s not real revenue. It’s next quarter’s homework. One RevOps team I know added “days to close by stage” into their forecast model. They realized 63% of late-stage pipeline wouldn’t close in time based on historical cycle length. The result? They re-weighted forecastable revenue by stage age × velocity. Forecast accuracy jumped 21% in two quarters. 2. Use behavioral signals, not just stage tags Stop assuming every Stage 4 opp has a 60% chance of closing. Start tagging based on buyer actions - not rep motion. What to track: - Was an economic buyer involved in the last call? - Did the buyer ask about implementation timeline? - Has procurement been looped in? - Are multiple stakeholders engaged and documented? Deals with 3+ of these signals close 2 - 3x more often. AND they close faster. Build a behavioral scoring model and overlay it on top of your CRM stages. 3. Build pipeline coverage by real math Forget the “3x coverage” rule of thumb. If your conversion rate from Stage 2 to Close is 18%, and your quarterly target is $1M, you don’t need $3M in pipeline. You need $5.56M in qualified opps. Idea: A CRO we work with built a stage by stage conversion model with time-based decay curves. They found that 22% of their pipeline had aged out of viable range, and 19% of Stage 1 deals had <5% chance of conversion. So they cut their pipeline headline by 41% - and finally forecasted accurately for the first time in six quarters. tl;dr = Forecasting isn’t about hope. It’s judgment × math × motion. If you’re still forecasting based on pipeline stage alone, you don’t have a sales process. You have a spreadsheet-shaped fantasy. And fantasy doesn’t hit number.

In Supply Planning, even a "perfect" forecast can quietly destroy your service levels. Let me share a scenario I encountered during a diagnostic review at a manufacturing plant. Two SKUs.   Both missed forecast by 5 units.   One had a weekly volume of 10. The other moved 1000 units. The system showed a stellar overall forecast accuracy.   But on the ground? One product constantly ran out of stock, while the other sat untouched in the warehouse. What was going on? The metric used—MAPE—told a lopsided story. It exaggerated the error for the smaller SKU and almost ignored the impact on the high-runner. The bias? Not even tracked. We replaced MAPE with MAE + Bias.   The moment we did, patterns emerged. We saw when we were consistently over-forecasting one SKU and under-forecasting another. The team adjusted safety stocks, demand drivers, and even supplier lead times accordingly. The result?   Lower inventory, better service levels, and more trust in the numbers. Because in supply chain, real accuracy isn’t about how close you look—it’s about how well you perform.

Why you shouldn't be using measures that sum across the errors of different time series/products (like MAE, MSE, WAPE, MAE% and others) to measure your forecasting performance... "Children usually learn at an early age the dangers of adding apples to orange (or apples to bananas or whatever). Yet in later life, we often fail to remember this maxim." (Chatfield, 1988) "The scale of the data often varies considerably among series. Series with large numbers might dominate comparisons. Despite this problem, the need for unit-free measures was not widely appreciated in the early 1980s" (Armstrong & Collopy, 1992) "There are some commonly used accuracy measures whose scale depends on the scale of the data. These are useful when comparing different methods applied to the same set of data, but should not be used, for example, when comparing across data sets that have different scales." (Hyndman & Koehler, 2006) 35+ years after the first of these references, still forecasting practitioners and consultants fail to understand this simple principle. Instead of summing the non-scaled errors and metrics across products (i.e., summing up apples with oranges), you should be using unit-free/scale-independent error metrics. My preference is to use relative measures (where the division takes place before summarising across products) which directly show the forecast-value-added (FVA) over a benchmark. It does not matter if some of these metrics (like WAPE or MAE%) apply scaling after summing across the units of products. At that point, the results are already non-sensible. References Chatfield C. (1988) "Apples, oranges and mean square error", International Journal of Forecasting, 4(4), 515-518. Armstrong J.S. & Collopy F. (1992) "Error measures for generalizing about forecasting methods: Empirical comparisons", International Journal of Forecasting, 8(1), 69-80. Hyndman R.J. & Koehler A.B. (2006) "Another look at measures of forecast accuracy", International Journal of Forecasting, 22(4), 679-688.

95% That's our current median forecast accuracy across all Recast clients. We update it weekly on our website because if you're going to trust a model with million-dollar decisions, you deserve to know if it actually works. Here's how we calculate it: Every week, we take versions of Recast models from 30, 60, or 90 days ago – before they saw recent data – and ask them to make a forecast. Given some amount of marketing spend per channel, how much of a KPI (like revenue, acquisitions, or app downloads) will be driven? Then, we check if these forecasts are right by comparing them to the outcomes our clients realized over these time horizons. We do this across hundreds of model refreshes using a metric called CRPS (Continuous Ranked Probability Score) that measures two things: 1️⃣ Did the forecast get close to reality? 2️⃣ Was it appropriately confident (not too wide, not too narrow)? It’s almost like hurricane forecast tracking. A good model doesn't just predict where the storm will go – it also shows an appropriate cone of uncertainty. CRPS scores both the accuracy and the uncertainty of our models. We run these tests on over 1,000 production models every month. Not because we enjoy the math, but because it’s a critical way for our clients to know if their models are worth trusting. A model that can't predict the future on data it hasn't seen is not worth all that much. That's why this number sits front and center in every Recast dashboard. Build trust through validation, then make decisions. Not the other way around. To learn more about Recast, check us out here: https://lnkd.in/eUpbB2BR

𝗨𝗻𝗱𝗲𝗿𝘀𝘁𝗮𝗻𝗱𝗶𝗻𝗴 𝗙𝗼𝗿𝗲𝗰𝗮𝘀𝘁 𝗔𝗰𝗰𝘂𝗿𝗮𝗰𝘆 𝗖𝗮𝗹𝗰𝘂𝗹𝗮𝘁𝗶𝗼𝗻 "𝗜𝗳 𝘆𝗼𝘂 𝗰𝗮𝗻’𝘁 𝗺𝗲𝗮𝘀𝘂𝗿𝗲 𝗶𝘁, 𝘆𝗼𝘂 𝗰𝗮𝗻’𝘁 𝗶𝗺𝗽𝗿𝗼𝘃𝗲 𝗶𝘁!" – This holds true for demand forecasting in supply chain planning. A good forecast minimizes stockouts, avoids overstocking, and improves service levels. But how do we measure forecast accuracy? 1️. 𝗠𝗲𝗮𝗻 𝗔𝗯𝘀𝗼𝗹𝘂𝘁𝗲 𝗘𝗿𝗿𝗼𝗿 (𝗠𝗔𝗘) – Measures Overall Forecast Error in Units: MAE calculates the average difference between actual and forecasted demand, showing the true magnitude of errors. 𝗠𝗔𝗘= 𝟭/𝗻 ∑〖𝗮𝗯𝘀 (𝗔𝗰𝘁𝘂𝗮𝗹𝘀-𝗙𝗼𝗿𝗲𝗰𝗮𝘀𝘁)〗 Example: If actual demand = 100, 150, 200 and forecast = 90, 160, 180, then: MAE = (∣100−90∣+∣150−160∣+∣200−180∣)/3=13.33 units Pros: > Easy to interpret (units of demand) > Useful for comparing multiple forecasts Cons: > Doesn’t indicate if errors are consistently positive or negative > Doesn’t penalize large errors more than small ones 2️. 𝗠𝗲𝗮𝗻 𝗔𝗯𝘀𝗼𝗹𝘂𝘁𝗲 𝗣𝗲𝗿𝗰𝗲𝗻𝘁𝗮𝗴𝗲 𝗘𝗿𝗿𝗼𝗿 (𝗠𝗔𝗣𝗘) – Measures Forecast Accuracy in % : MAPE expresses forecast error as a percentage of actual demand, making it easy to compare across products or industries. 𝗠𝗔𝗣𝗘= 𝟭/𝗻 ∑ (𝗮𝗯𝘀 (𝗔𝗰𝘁𝘂𝗮𝗹𝘀-𝗙𝗼𝗿𝗲𝗰𝗮𝘀𝘁))/𝗔𝗰𝘁𝘂𝗮𝗹𝘀  Example Calculation: MAPE = (∣100−90∣ / 100 + ∣150−160∣ / 150 + ∣200−180∣ / 200)) / 3 × 100 = 8.89%  Pros: > Expressed in percentage → easy to understand > Works well when comparing multiple product forecasts Cons: > Skews results when demand is low > Can’t be used for zero-demand periods 3️. 𝗠𝗲𝗮𝗻 𝗦𝗾𝘂𝗮𝗿𝗲𝗱 𝗘𝗿𝗿𝗼𝗿 (𝗠𝗦𝗘) – Penalizes Large Forecast Errors MSE squares the error values before averaging them, making larger errors more impactful. 𝗠𝗦𝗘= 𝟭/𝗻 ∑〖(𝗔𝗰𝘁𝘂𝗮𝗹𝘀-𝗙𝗼𝗿𝗲𝗰𝗮𝘀𝘁)〗^𝟮 Example Calculation: MSE = ((100−90)^2 + (150−160)^2+ (200−180)^2) / 3 = 200  Pros: > Penalizes large errors more than small ones > Differentiates between "small" and "big" forecasting mistakes Cons: > Squaring the errors amplifies outliers > Harder to interpret since it’s not in demand units 4️. 𝗥𝗼𝗼𝘁 𝗠𝗲𝗮𝗻 𝗦𝗾𝘂𝗮𝗿𝗲𝗱 𝗘𝗿𝗿𝗼𝗿 (𝗥𝗠𝗦𝗘) – More Interpretable than MSE RMSE is simply the square root of MSE, bringing it back to the original demand units. RMSE= √(MSE) RMSE = √(200) = 14.14  Pros: > More interpretable than MSE (same unit as demand) > Useful when large errors are critical Cons: > Still penalizes large errors more than small ones > Harder to compare across different product categories The key is to continuously monitor, refine, and adapt based on insights from these metrics. Because in supply chain planning, the goal isn’t a perfect forecast—it’s a smarter, more resilient one! #SupplyChain #Demandforecasting #InventoryManagement #DemandPlanning #CostOptimization #Logistics #Procurement #InventoryControl #LeanSixSigma #Cost #OperationalExcellence #BusinessExcellence #ContinuousImprovement #ProcessExcellence #Lean #OperationsManagement

Because wrong math equals tragedy in demand and supply planning... This infographic shows 12 fundamental calculations: ✅ #1 - MAPE (Mean Absolute Percentage Error) ❓ What: measures the average percentage error in forecasts 🧮 Calculation: [Sum of (absolute forecast errors / actual sales for each time period) / Total number of periods] X 100 ✅ #2 - WMAPE (Weighted Mean Absolute Percentage Error) ❓ What: provides a more balanced view of forecast accuracy by weighting errors 🧮 Calculation: [Sum of (absolute forecast errors / actual sales for each time period) / Sum of Actual sales] X 100 ✅ #3 - Forecast Bias ❓ What: identifies whether forecasts consistently overestimate or underestimate demand 🧮 Calculation: Sum of (forecast - actual) / Sum of Actual sales ✅ #4 - FVA (Forecast Value Added) ❓ What: measures the improvement (or deterioration) in forecast accuracy after applying a specific forecasting process or intervention 🧮 Calculation: FVA=Baseline Error−New Forecast Error 👉 where Baseline Error = forecast error from a reference method; New Forecast Error = forecast error from new method ✅ #5 - Demand Variability (Coefficient of Variation) ❓ What: indicates the consistency or volatility of demand 🧮 Calculation: Demand Variability= (σ / μ ) X 100 👉 where σ = Standard deviation of demand; μ: Average demand ✅ #6 - Promotional Lift Factor ❓ What: evaluates the impact of promotions on demand 🧮 Calculation: Lift Factor = Promotional Demand / Baseline Demand ✅ #7 - OTIF (On Time In Full) ❓ What: shows how many orders are delivered on time in full 🧮 Calculation: (Number of On Time In Full Deliveries / Total Number of Deliveries) X 100 ✅ #8 - Inventory Turnover ❓ What: indicates how fast or slow inventory is moving 🧮 Calculation: (Cost of Goods / Average inventory) X 100 ✅ #9 - Safety Stock ❓ What: determines the buffer stock needed to handle demand variability 🧮 Calculation: Z×σ×√Lead Time 👉 where Z = Z-score (based on the desired service level); Σ = Standard deviation of demand or lead-time demand ✅ #10 - Obsolescence Ratio ❓ What: tracks the portion of inventory that is no longer usable 🧮 Calculation: (Obsolete Inventory / Total Inventory) X 100 ✅ #11 - Production Yield ❓ What: measures production efficiency 🧮 Calculation: (Good Units Produced / Total Units Produced ) X 100 ✅ #12 - Capacity Utilization ❓ What: tracks how much of available capacity is being used 🧮 Calculation: (Actual Output / Maximum Capacity) X 100 Any others to add?