Software & AI · Document intelligence · Written and maintained by Haink’s AI adoption team · Updated August 2026 · 14 min read
Agents in repeatable processes: what autonomy costs you
What this page argues
- Autonomy and repeatability pull in opposite directions, and the conflict is arithmetic: a step that reproduces 95% of the time reproduces a six-step chain 74% of the time, and a twelve-step chain 54%.
- The conflict is not a reason to avoid agents. It is a reason to confine them — to steps where the path is genuinely unknown and a person checks the result.
- The architecture that survives is hybrid: a deterministic process skeleton with agents inside individual steps, bounded and logged.
- Log the intent, not only the outcome. Which steps were chosen and why is the part that makes a decision reconstructable, and the part everyone omits until an auditor asks.
- Most agent pilots do not scale, and the reasons are organisational and architectural rather than model quality.
Demand tripled. Practice did not follow.
Two thirds of enterprise document-processing initiatives are now evaluating an agentic approach rather than a conventional extraction-and-rules pipeline. Two years ago it was under a quarter — a threefold rise in intent inside twenty-four months.
Nothing comparable happened to the body of practice underneath it. Survey the material a decision-maker can actually read and it divides into two piles: how to build an agent — frameworks, orchestration patterns, tool calling, evaluation harnesses — and what agents could do for you, which is a capability list. Both are written from the perspective of someone building agents. Almost nothing is written from the perspective of someone who will have to explain, eighteen months later, why a particular file was handled the way it was — to an auditor if they are unlucky, and to a supplier or a losing bidder far more often.
That gap is the reason this page exists, and it is also why the failure mode is predictable. A team reads architecture material, builds something that demonstrates beautifully, and meets the question it was never designed for at the internal audit gate. The technology is not what failed. Nobody told them which question was coming.
A fast-moving field with thin practice produces a specific hazard: the confident recommendation. When a market grows threefold in two years, the volume of advice grows with it and the amount of production experience behind that advice does not. Treat any source telling you where agents belong — including this one — as a claim to be checked against your own audit requirement rather than a conclusion to adopt.
The conflict, stated plainly
An agent is defined by the thing that makes it useful: it decides what to do next. Given a document that is missing a page, it might re-read, fetch a related record, request the page, or escalate — and which of those it does is determined at runtime, not written down in advance.
A great many business processes are defined by the opposite property. They require that the same input produce the same output — and that a specific decision from eight months ago can be reconstructed exactly as it was made.
It is tempting to file that requirement under “compliance” and assume it applies to banks and pharmaceutical manufacturers. It does not. Sameness is a requirement wherever a decision can be challenged by the person it affects, and that is most of commercial life.
| Process | Why the same answer is required | What breaks without it |
|---|---|---|
| Comparing tender submissions | Bidders have to be assessed on equal terms | The award is challenged and the procurement re-run |
| Checking work orders and inspection reports | The same defect must be caught every time it appears | A miss the contractor is answerable for |
| Matching invoices against purchase orders | The same discrepancy must be treated the same way | A dispute with the supplier you cannot win on the record |
| A credit decision | A supervisor requires it | A regulatory finding |
Only the last row involves a regulator. The other three are ordinary procurement, maintenance and finance work in companies with no compliance function at all — and in each of them an inconsistent answer is expensive for the same underlying reason: someone disagrees with the outcome and asks how it was reached.
The regulated case is the strictest, because there the requirement is written down and someone is empowered to enforce it. It is not the only case. Wherever a decision gets contested, non-determinism costs money whether or not a rulebook exists — and the contest is usually with a supplier, a bidder or a contractor rather than an authority.
These two requirements are in direct conflict, and no amount of prompt engineering resolves it. Lowering the temperature narrows the distribution; it does not make the sequence deterministic. Pinning the model version freezes one input to the decision, not the decision. Adding a “be consistent” instruction is not a control. The conflict is structural, so the answer has to be architectural.
What follows from that is not “do not use agents”. It is that autonomy has to be spent where it buys something and withheld where it costs you the audit — which requires knowing which is which.
The conflict is arithmetic, not rhetoric
“Agents are less reproducible” is easy to say and easy to wave away. Put a number on it and it stops being a matter of opinion, because reproducibility compounds down the chain exactly the way character errors compound across a field.
Take a single agent step that, given identical input, chooses the identical action p of the time. Over a chain of n steps, the probability that the whole sequence repeats is pn.
Read the 95% row, because that is roughly where a well-built agent step sits and it sounds excellent. Over six steps the same agent reproduces its own sequence three times in four. Over twelve, barely half the time. An auditor does not ask whether step four was consistent; they ask whether this decision would happen again — and that is the whole chain.
Two consequences fall out of the table, and both are design rules rather than opinions:
- Short chains or none. Every additional autonomous step multiplies the reproducibility loss. An agent confined to one step of a deterministic process sits in the leftmost column permanently; an agent orchestrating the whole flow does not.
- Per-step consistency is the wrong metric to report. It is the number that looks good in a demo and the number that misleads. Report chain-level reproducibility on a fixed test set, or report nothing.
The cost that nobody models
A deterministic pipeline has a fixed unit cost: the same steps every time, so the invoice is predictable. An agent's cost scales with how many steps it decides to take, and that distribution has a tail.
Budgets get built on the average because the average is what the pilot reports. Then production delivers the tail — the awkward cases that trigger the most tool calls are also the ones that arrive in volume at month end — and the run-rate lands somewhere between two and three times the forecast. This is a recoverable problem if the cost per case is instrumented from day one, and an unpleasant surprise if it is not. It is also, in our experience, the second most common reason an agentic pilot is quietly shelved after the reproducibility one.
Where an agent earns its place, and where it breaks things
- Exception handling where the resolution path depends on what turns out to be wrong — and nobody can enumerate the cases in advance.
- Gathering context across systems, where the next lookup depends on the previous answer.
- Producing a draft that a named person reviews before it has any effect.
- Routing with a stated reason, where the reason is the deliverable and the routing is reversible.
- Any step that must be reproducible — the same file, the same answer, permanently.
- Any decision that gets contested — by a bidder, a supplier, a contractor or a regulator — where “the model chose to check this next” is not an explanation.
- Final actions without review anywhere the decision gets contested — payment, approval, decline, release, award.
- Anywhere the error cost is asymmetric, because a variable path makes the tail unbounded.
The pattern in the left column is that a human either checks the output, or the cost of an imperfect one is low. The pattern in the right column is that neither is true. That single test resolves most cases without needing a framework opinion.
The architecture that holds
The version we build, and the reason it survives review:
Three properties make this work, and each of them is a constraint rather than a feature.
The agent cannot change the process. It runs inside one box. It cannot decide that validation is unnecessary today, because the sequence is not its to choose — only the work within its own step is.
Its output is schema-constrained. Not free text handed to the next stage, but a structured result the deterministic layer can validate before acting on. If it fails validation, that is an exception, not a silent pass-through.
Its intent is logged, not just its answer. Which is the next section, because it is the part that gets skipped.
What to log so a decision can be rebuilt
Four things. Cheap on day one; expensive and incomplete if added later, because the period before you started logging is simply unrecoverable.
| What | Why it is needed |
|---|---|
| Inputs as received | The file as it arrived, not as it was later corrected. Otherwise the reconstruction runs on different evidence from the original decision. |
| Model and prompt versions | A pinned version is the only thing that makes a re-run meaningful. With a hosted endpoint that can move without notice, this may be unobtainable — which is an argument for self-hosted open weights in regulated work. |
| The intent trace | Which steps the agent chose, in what order, on what basis. This is the item that distinguishes a reconstructable decision from a stored answer, and it is the one almost always missing. |
| The human review step | Who reviewed it, what they were shown, what they approved. Without this the record proves the machine acted and nothing about accountability. |
The storage cost of an intent trace is trivial next to the cost of not having one during a regulatory conversation. The same argument applies to evidence chains in adjacent processes — see how AI claims processing works and due diligence document review, where the reconstruction requirement arrives from different directions and lands in the same place.
When the process is high-risk, this stops being optional
Everything above holds wherever a decision is contested. In one situation it stops being commercial prudence and becomes a legal obligation — the strictest version of the same requirement, not a different one.
If the process falls inside Annex III of the EU AI Act — creditworthiness assessment of natural persons, or risk pricing in life and health insurance — then human oversight, record-keeping and reproducibility are obligations rather than preferences, binding from 2 December 2027. An agent choosing its own sequence inside such a step is not merely hard to audit; it is hard to reconcile with an obligation to explain a specific outcome to a specific person.
The boundary itself, including what stays outside it, is worked through in AI underwriting and the EU AI Act high-risk line. The practical consequence for agent design is simple: put the agent on the preparing side of the line, never on the deciding side.
Why agentic pilots do not scale
Reported figures cluster around a stark ratio: roughly four in five enterprises have an agent pilot running, and about one in seven has scaled an agent to organisation-wide operational use. The gap is not explained by model quality.
- Non-determinism inside an auditable loop. The pilot passes because nobody audited it. The production gate is internal audit, and it asks the reproducibility question the demo never faced.
- No ground-truth set. Without an agreed labelled sample, “is it good enough” becomes an argument between opinions, and the project stalls at approval.
- Integration with systems built for predictable callers. Downstream systems assume a fixed call pattern. An agent that varies its sequence breaks assumptions nobody documented.
- Cost per case is variable and nobody modelled it. A pipeline has a predictable unit cost; an agent that sometimes takes twelve steps does not. Finance discovers this in month three.
- No owner for the last mile. The general pattern, agentic or otherwise, is in why AI pilots fail.
How to decide, in one pass
For each step in your process, three questions. If any answer is on the right, the step is deterministic.
| Question | Agent is viable | Keep it deterministic |
|---|---|---|
| Can you write down the path in advance? | No — the cases are genuinely open | Yes, or mostly yes |
| Does a person check the output before it takes effect? | Yes | No |
| Will you have to explain this exact outcome later? | No, or only in aggregate | Yes, case by case |
Most document processes have a handful of steps in the left column and a majority in the right. A vendor proposing agents across the whole flow has either not asked these questions or has decided the answers do not matter — and the second is worse than the first.
Frequently asked questions
What is agentic document processing?
A document pipeline in which a model decides what to do next rather than following a fixed sequence: it can choose to re-read a page, fetch a related record, ask for a missing document or escalate, and the order of those steps is not written in advance. That is the whole difference from conventional automation, and it is both the value and the problem — the same property that lets it handle an unforeseen case prevents it from guaranteeing that an identical input produces an identical output.
Can an AI agent be used in an audited process?
Inside a step, yes. As the process, no. An audit — or a supplier disputing an outcome — asks two things an autonomous sequence cannot promise: that the same input yields the same output, and that a specific decision can be reconstructed months later exactly as it was made. The workable pattern is a deterministic process skeleton with agents confined to individual steps that have fixed boundaries, defined inputs and outputs, and logging of the intent rather than only the result.
What is the difference between agentic AI and rules-based automation?
Rules-based automation executes a path someone wrote down; an agent selects a path at runtime. Rules give you reproducibility, cheap operation, loud failure and an explanation by construction. An agent gives you coverage of situations nobody anticipated, at the cost of variable sequences, variable cost per case and an explanation that has to be reconstructed from logs. Neither is better in general — they fail in opposite directions, which is why most durable systems use both.
What has to be logged for an agent decision to be reconstructable?
Four things, and the third is the one usually missed. The inputs as received; the model and prompt versions in effect; the intent trace — which steps the agent chose, in what order, and on what basis — not merely the final answer; and the human review step with the reviewer's identity and what they saw. Building this in at the start costs very little. Retrofitting it after a year of operation is expensive and produces a partial record for the period before it existed.
Where do agents genuinely earn their place in document work?
Where the path is genuinely unknown in advance. Exception handling, where the resolution depends on what turns out to be wrong. Gathering context across several systems, where the next lookup depends on the last answer. Producing a draft for a person to check. And routing with a stated reason. What these share is that a human either checks the output or the cost of an imperfect one is low.
How much less reproducible is an agent, in numbers?
Reproducibility compounds down the chain. If a single agent step chooses the identical action on identical input p of the time, an n-step chain repeats with probability p to the power n. A step that is 95% self-consistent — which sounds excellent — reproduces a six-step chain 74% of the time and a twelve-step chain 54% of the time. An auditor does not ask about step four; they ask whether this decision would happen again, and that is the whole chain. Two rules follow: keep autonomous chains short or absent, and report chain-level reproducibility rather than per-step consistency.
Why does agentic cost per case overrun the budget?
Because a deterministic pipeline has a fixed unit cost while an agent's cost scales with the number of steps it decides to take, and that distribution has a tail. With a mean of five steps and a 95th percentile of twelve, budgeting on the average understates the tail by roughly two and a half times — and the awkward cases that trigger the most tool calls tend to arrive in volume rather than evenly. Instrument cost per case from the first day and the problem is manageable; discover it in production and it is the second most common reason an agentic pilot is quietly shelved.
Why do agentic pilots fail to scale?
Reported figures put the share of enterprises that have scaled an agent to organisation-wide operational use at around one in seven, against roughly four in five that have a pilot running. The recurring causes are not model quality: non-determinism inside a process that has to be auditable, no ground-truth set to argue against, integration with systems that were not designed for a caller which behaves differently each time, and no owner for the last mile. The model is rarely what fails.
This page describes engineering and governance practice. It is not legal advice, and where a process may fall under the EU AI Act or an equivalent regime, the classification question belongs with your own counsel.
Related Resources
- Agentic Automation for Document Workflows — where we place agents, and where we refuse to
- AI Underwriting and the EU AI Act High-Risk Line — where human oversight stops being optional
- Document Data Capture Methods: Which to Choose
- Due Diligence Document Review
- LLM Applications & RAG
Deciding where the agent goes is the design
A deterministic skeleton with agents confined to the steps where the path is genuinely unknown — bounded, schema-constrained, and logged down to the intent.
Sources. The shift from under a quarter to roughly two thirds of enterprise document initiatives evaluating agentic approaches is attributed to Gartner’s 2025 intelligent document processing research; we have seen it only through secondary reporting and cite it as such. Pilot-to-scale figures — roughly four in five enterprises piloting, about one in seven scaled to organisation-wide use — come from 2026 industry surveys and vary by definition of “scaled”; treat the ratio rather than the decimal as the finding. EU AI Act dates are from Regulation (EU) 2024/1689 as amended by Regulation (EU) 2026/1744.
Reviewed: August 2026.