Software & AI · Operations · Written and maintained by Haink’s AI adoption team · Updated August 2026 · 8 min read
MLOps: Getting Machine Learning Models to Production
Most machine-learning models never make it out of a notebook. MLOps is the engineering discipline that gets them into production and keeps them working as data and the world change. The reasons models fail to ship are almost always operational — not modeling — which is why MLOps, rather than a better algorithm, is usually what unblocks a stuck project.
Key takeaways
- MLOps covers reproducible training, CI/CD for models, deployment, monitoring, drift detection, retraining and governance.
- Models usually fail to reach production for engineering reasons, not modeling ones.
- The “85% never ship” figure everyone quotes is folklore — it traces to a 2017 conference remark about big data projects and a 2019 trade-press article, not to research on ML models.
- Label latency, not compute, sets your retraining cadence. If the truth about a prediction arrives ninety days later, you cannot retrain monthly on truth however much GPU you own.
- Unlike normal software, an ML model can silently get worse as data drifts — monitoring is essential.
- A retraining pipeline ready to go is what keeps a model trustworthy over time.
- Training and inference run on GPUs; MLOps includes operating that hardware.
On this page
About that 85% statistic
Almost every article on this subject opens with a number: 85% of models never reach production, or 87% of data science projects fail. It is worth knowing where those came from before building a business case on them.
| The figure | Where it actually comes from | What it was about |
|---|---|---|
| 85% | A remark by a Gartner analyst in 2017, repeated ever since | Big data projects — not ML models, not deployment rates |
| 87% | A VentureBeat article, July 2019 | Data science projects reaching production, in the pre-foundation-model era |
Neither is recent, neither is peer-reviewed, and neither measured what the sentence quoting it usually claims. They are seven to nine years old and describe a period before managed model APIs, before open-weight models were competitive, and before most of the tooling in this article existed.
That does not make the underlying observation wrong — a lot of models do sit in notebooks, and the reasons are operational rather than mathematical. It makes the number wrong to cite. If you need a figure for a business case, measure your own: how many models your team has trained in two years, and how many are serving traffic today. That ratio is knowable, specific to you, and defensible in a room where someone asks where it came from.
What MLOps covers
Seven areas, and the tools listed are examples of each category rather than recommendations — the right choice depends on scale, on whether you are self-hosting, and on what your platform team already runs.
| Area | Purpose | Typical tooling |
|---|---|---|
| Reproducible training | Versioned data, code and models, so a result can be rebuilt and trusted a year later | DVC, MLflow, Weights & Biases |
| Feature management | One definition of a feature shared by training and serving, so the two cannot silently diverge | Feast, Tecton, or a disciplined warehouse |
| Orchestration | Pipelines that run on a schedule or a trigger, with retries and lineage | Airflow, Dagster, Kubeflow |
| CI/CD for models | Automated testing, validation and promotion from experiment to production | A model registry plus your existing CI |
| Deployment and serving | Reliable serving at the required latency and scale, on cloud or on-premises GPUs | KServe, Triton, vLLM for LLM serving |
| Monitoring | Accuracy, latency and input quality in production — not just at training time | Evidently, Great Expectations, your observability stack |
| Governance | Audit trails, access control and documentation for compliance and accountability | Model registry, lineage, and whatever your auditor accepts |
The one row people skip is the one that bites. Feature management sounds like over-engineering until training and serving compute the same feature slightly differently — a different time window, a different null handling — and the model performs worse in production than in every test, for reasons nobody can find. Training-serving skew is the most common silent failure in production ML, and it is a plumbing problem with a plumbing fix.
Why models fail to reach production
A model that scores well offline still fails in production when training isn't reproducible, when there's no reliable way to deploy and roll back, when nobody is watching for drift, or when retraining is a manual scramble. These are engineering gaps, not modeling gaps. Teams often respond by tuning the model further when the real fix is the operational scaffolding around it.
Label latency sets your retraining cadence
“How often should we retrain?” gets answered with a habit — quarterly, usually — when it has an arithmetic answer and a hard constraint, and most teams know neither.
The arithmetic first. If a model decays at d points of accuracy a month and you retrain every T months, the average degradation you are living with is d × T / 2. At half a point a month:
| Retraining interval | Average accuracy you are running at | Retrains per year |
|---|---|---|
| 12 months | 3.00 points below fresh | 1 |
| 6 months | 1.50 points | 2 |
| 3 months | 0.75 points | 4 |
| 1 month | 0.25 points | 12 |
Going from annual to quarterly removes 75% of the average degradation for four times the retraining cost. Going from quarterly to monthly removes a further 67% for three times more. The returns fall off steeply, and where you stop depends on what a point of accuracy is worth in your process — which is a business number, not an ML one.
But none of that is usable until you know d, and d can only be measured against labelled production data. Which brings the constraint nobody puts in the diagram:
A team with ninety-day label latency cannot retrain monthly on ground truth however much GPU it owns; it can only retrain monthly on proxies, and proxies drift in ways nobody notices until the real labels land. That is a design constraint, not an inconvenience, and it belongs in the first architecture conversation rather than the first incident.
It also points at an advantage that goes unremarked in document work: a human-in-the-loop document system generates a correct label every time a reviewer accepts or corrects a field. Label latency is effectively zero, the training set grows for free as a by-product of the process running, and decay is measurable continuously rather than annually. That is one of the quieter arguments for the review-queue architecture described in how AI document processing works.
Monitoring and drift are the real ongoing work
Unlike conventional software, an ML model can silently get worse as the world drifts away from its training data — a fraud model decays as fraud evolves, a demand model decays as behavior shifts. Production monitoring of inputs, outputs and quality metrics, with alerts and a retraining pipeline ready to go, is what keeps a model trustworthy over time. Two kinds of drift matter: data drift (the inputs change) and concept drift (the relationship between inputs and the right answer changes).
MLOps for LLMs (LLMOps)
LLM applications add their own operational needs on top of classic MLOps: prompt and version management, evaluation sets that score answer quality, guardrails against prompt injection, and cost and latency monitoring per request. The principle is the same — measure quality in production and have a path to improve it — but the tools and failure modes differ from traditional ML.
Three differences matter operationally. There is often no retraining loop at all — you change prompts, retrieval and rules rather than weights, which makes the release cycle faster and the versioning discipline more important, not less. The model can change underneath you if it is hosted, which is a category of incident traditional MLOps never had to handle; the consequences where that is unacceptable are in document extraction under validation. And quality is not a single scalar: retrieval and generation have to be scored separately, because an aggregate hides which half is failing — set out in how to build a production RAG system.
MLOps and infrastructure
Training and inference run on GPUs, and MLOps includes operating that hardware: scheduling, queuing, and a runtime sized to the workload rather than to the model.
Whether that hardware should be yours is a question with a computed answer rather than a philosophical one, and the answer is usually no on cost alone — the crossover against a hosted API sits near 340 tokens per second sustained around the clock, worked through in on-premises versus cloud LLM deployment. Training is the exception that changes the shape: it is bursty by nature, which argues for renting capacity, while inference is steady, which is what eventually argues for owning it. Many estates end up renting for training and owning for serving, and that split is a conclusion rather than a compromise.
Where the data cannot leave the perimeter at all, the whole loop — training, serving, monitoring and retraining — runs inside it, and we supply the hardware alongside the software under one contract. See private AI infrastructure.
Building AI software on your own infrastructure?
Model, pipeline and GPUs under one contract — tell us the use case and we'll scope it.
What to read next
- How to build a production RAG system — the LLM equivalent, where evaluation replaces retraining
- On-premises vs cloud LLM — whether the hardware under all this should be yours, computed
- How much does custom AI cost? — where monitoring and retraining sit in the budget
- From blueprint to implementation — how a specified system becomes an operated one
Related Resources
Frequently Asked Questions
What is MLOps?
MLOps is the set of practices that take machine-learning models from experiment to reliable production — reproducible training, CI/CD for models, deployment, monitoring, drift detection, retraining and governance.
Why do machine-learning models fail to reach production?
Usually because of engineering gaps, not modeling: training isn't reproducible, there's no reliable deployment and rollback, nobody monitors for drift, and retraining is manual. MLOps closes those gaps.
Is it true that 85% of ML models never reach production?
The figure is folklore rather than evidence. The 85% traces to a remark by a Gartner analyst in 2017 about big data projects, not about ML models or deployment rates; the related 87% comes from a VentureBeat article in July 2019. Both predate managed model APIs, competitive open-weight models and most current tooling. The underlying observation — that many models never leave a notebook, for operational rather than mathematical reasons — is sound. The number is not, and a business case built on a borrowed statistic collapses when somebody checks it. Measure your own ratio: models trained in two years versus models serving traffic today.
How often should we retrain a model?
Two things decide it. The arithmetic: if a model decays at d points a month and you retrain every T months, average degradation is d times T divided by two — so at half a point a month, annual retraining leaves you three points below fresh, quarterly leaves 0.75, and monthly leaves 0.25. Returns fall off steeply, and where you stop depends on what a point of accuracy is worth in your process. The constraint: label latency. You cannot retrain on truth faster than the truth arrives, so a fraud model with a ninety-day chargeback window cannot be retrained monthly on ground truth however much compute you have.
What is training-serving skew?
When a feature is computed one way during training and slightly differently at serving time — a different time window, different null handling, a different join — the model performs worse in production than in every test, and the cause is invisible in the model itself. It is the most common silent failure in production ML and the reason a feature store, or at least one shared feature definition, earns its keep despite looking like over-engineering.
What is model drift?
Drift is when production data moves away from the data a model was trained on (data drift) or the input-to-answer relationship changes (concept drift), silently degrading accuracy. Monitoring and a retraining pipeline catch and correct it.
What is the difference between MLOps and LLMOps?
LLMOps applies MLOps principles to LLM applications, adding prompt and version management, answer-quality evaluation, guardrails against prompt injection, and per-request cost and latency monitoring.
Can MLOps run on-premises?
Yes. Training, inference, monitoring and retraining can all run on private or air-gapped infrastructure, with right-sized GPUs supplied alongside the software.
Sources and scope. The 85% figure is attributed to a Gartner analyst's 2017 remark about big data projects and the 87% to a VentureBeat article of July 2019; both are cited here to show their provenance, not as evidence. The retraining table is arithmetic from a stated decay rate, not a measurement — the average degradation under a retraining interval is decay rate times interval divided by two, and your own decay rate has to be measured against labelled production data. Label-latency examples are typical rather than universal and vary by product and jurisdiction. Named tools are examples of each category, not recommendations.
Reviewed: August 2026.