Reframing a Forecasting Problem: When Accuracy Was the Wrong Target
A pharmaceutical distributor with 11,000 SKUs across four regional depots asked for one thing: improve demand forecast accuracy from its 34% MAPE baseline. Envion’s first deliverable was an argument that accuracy was not the metric that mattered — the cost of forecast error was wildly asymmetric, and no accuracy metric captured it. A segmented, cost-optimized approach with predictive distributions replaced the single point forecast, and the cost matrix built with supply chain and finance now runs planning policy independently of any model.
The challenge
The client's existing forecast — a moving average maintained in Excel by three planners — ran at roughly 34% MAPE. They wanted a machine learning model to bring it down and had benchmarked a vendor promising sub-20%.
Before any modelling, Envion assessed whether the underlying data could support a forecast at all. The history recorded shipments, not demand: every stockout in seven years appeared as low demand, teaching any model to repeat the mistake that hurt most — Envion reconstructed censored demand using order-line rejection logs the planning team had never connected to the forecast. SKU codes had changed across two ERP migrations, leaving 1,400 products with discontinuous history that looked like demand collapse. Regulatory formulary changes, generic entries and competitor withdrawals created structural breaks the commercial team knew about months ahead — knowledge that had never entered the forecasting process. And 41% of SKUs sold on fewer than 15% of days, where standard accuracy metrics behave badly and MAPE is undefined at zero.
Decision path
Envion's core finding: the cost of forecast error was wildly asymmetric, and no accuracy metric captured it. Under-forecasting a critical-care product meant a stockout, an emergency transfer, and in some cases a clinical escalation — cost measured in thousands per event plus contractual penalties. Over-forecasting a short-dated cold-chain product meant expiry write-off. Over-forecasting a stable high-volume generic cost almost nothing beyond carrying cost. A model optimizing symmetric error treats all three identically; the right objective was expected cost, not expected error.
Envion built a cost matrix with the client's supply chain and finance leads — per-SKU stockout cost, expiry cost, carrying cost and shelf life. This took three weeks and was, by the client's own account, the most contested part of the engagement — and the most valuable, because it forced explicit agreement on trade-offs previously made implicitly by whoever was on shift.
Envion contribution
Rather than one model, Envion prototyped a segmented approach, each segment measured against the cost objective on a held-out year: gradient-boosted models with calendar, promotion and formulary features for fast-moving stable SKUs (18% of SKUs, 71% of volume); Croston-family methods for intermittent SKUs — ML models were tested and performed worse, which Envion reported rather than buried; hierarchical borrowing from therapeutic class for new and short-history SKUs; and a planner override interface — not a model — for known structural breaks, with the commercial team's forward calendar as a required input.
Crucially, the output was a predictive distribution, not a point forecast. Safety stock was set per SKU at the service level that minimized expected cost given the cost matrix — high for critical-care, deliberately low for short-dated products where expiry dominated.
Delivery
Envion's final deliverable defined the production path explicitly so the client could budget it rather than discover it: a demand-uncensoring step in the data pipeline; the SKU mapping table maintained as a system of record with ownership assigned; structured intake from the commercial team for known forward events; quarterly retraining with drift monitoring on the cost metric rather than on accuracy; planner override logging so overrides become training signal instead of invisible corrections; and an estimated 5-month build — with a recommendation to run the model in parallel with the planners for a full quarter before it drove any purchase order.
Practical rules from the project: ask what each direction of error costs — if under-forecasting costs ten times over-forecasting, a model minimizing MAPE is actively working against you; check whether your history records demand or fulfilment — censored demand biases the forecast hardest exactly where being wrong is most expensive; and ask for a distribution, not a number — uncertainty is not a caveat on the answer, it is the answer.
Outcome and evidence
In the first year in production, fast-moving forecast MAPE fell from 34% to 21%, critical-care stockouts from 218 to 47 a year, expiry write-off from €3.4M to €1.9M, inventory holding value fell 12%, emergency inter-depot transfers fell from 1,140 to 390 a year, and planner time on manual forecasting dropped from roughly 60% to 20%.
Note the shape of it: inventory fell and stockouts fell, which a pure accuracy improvement would not have delivered — that came from allocating safety stock by cost rather than uniformly.
Client feedback
What the client says about this engagement
“I asked for a more accurate forecast and Envion spent the first three weeks telling me accuracy was the wrong thing to buy. I was irritated. Then they showed me that our best-performing SKUs by accuracy were also our worst by expiry cost, and that landed.
The cost matrix workshops were painful — getting supply chain and finance to agree on what a stockout of a critical-care product actually costs took four sessions — but that document now runs our planning policy independently of any model. And they told us plainly that ML performed worse than a simple method on 41% of our SKUs. A vendor selling us a model doesn't say that.”
Evidence gate. This page publishes only what Envion's project records and client disclosure permissions support. Outcomes are added once verified against a baseline, a measurement period, and an approved source.
FAQ
Questions about this case
Facing a similar challenge?
If you are about to buy forecast accuracy, discuss the cost of each direction of error with Envion first — the cost matrix is worth the exercise even if you never build a model.
Discuss a Similar ChallengeKeep exploring
Similar case studies
Executive Technology Leadership
Support for high-stakes product and AI decisions
Bring senior technology leadership into the business when the roadmap is unclear, delivery is at risk, an AI initiative needs stronger ownership, or the company needs an experienced technical voice before hiring a permanent CTO.
Discuss Interim CTO SupportCore responsibilities
- Align product and technology priorities with business goals and measurable outcomes.
- Review architecture, delivery risks, data foundations, security needs, and AI readiness.
- Lead internal teams and external partners through a practical execution plan.
- Clarify team structure, ownership, decision rights, and delivery cadence.
- Support investor, board, partner, and due-diligence conversations with credible technical judgment.
New experience
Prompt-to-Page — try it right here
Describe the landing page you want, in your own words. We turn it into a finished page and email you a private link in 5–10 minutes — no briefs, no calls, $0 to see the result.
- Describe what you want to create.
- We structure, write, and compose the page.
- You receive a private link when it is ready.
Start with a sentence — the interactive builder takes it from there.
Generate My PageSafe, respectful content only. No obligation.