AI Agent Worked Example — The Agent That Cost More Than the People
A worked example of an agent that worked and still lost money — how the loaded cost was calculated, where it actually went, and the four changes that made it cheaper than the process it replaced.
This is an illustrative example. The company, agent and figures are invented. The arithmetic is the point: an agent can succeed on every measure being watched and still cost more than what it replaced.
The situation#
A B2B software company built an agent to handle inbound support triage: classify the ticket, pull the customer's account history and recent errors, and draft a first response. A human approves every message before it is sent.
After two months it was working. Completion rate 78%, model spend $840 a month against a support team costing far more than that, and the support lead was pleased.
Then finance asked for cost per resolved ticket, and the number was $6.20 against $4.10 for the old fully-manual process.
Where the money actually was#
Model spend was never the question. Reconstructing the cost from the audit log and two weeks of observed handling time gave this:
| Volume | Human time each | Total monthly | |
|---|---|---|---|
| Approved as drafted | 2,510 | 40 sec | 28 hours |
| Edited before sending | 1,003 | 3.5 min | 59 hours |
| Handed over after the agent stopped | 397 | 9 min | 60 hours |
| Failed on a tool error | 83 | 11 min | 15 hours |
| Total human time | 162 hours | ||
| Model and tool spend | $840 |
The 480 tasks the agent did not finish consumed 75 hours — 46% of all human time, for 11.5% of the volume.
Those tasks were more expensive than if the agent had never touched them, because the handler had to read the transcript, work out how far it had got and what it had already told the customer, and then start. Handling a ticket from scratch took 6 minutes. Handling one the agent had abandoned took 9.
The trap in the numbers#
The measure everyone was watching was completion rate, and 78% looked healthy. It hid the structure entirely.
Cost is not proportional to failure rate. A task the agent completes saves about 5 minutes; a task it abandons costs about 3 extra. At those ratios roughly one abandoned task cancels the saving from two completed ones, which means an agent at 78% completion is barely breaking even — and this one was not, once the edits were counted.
Nobody had calculated this because the model spend was so obviously small that cost felt like a solved question.
What the transcripts showed#
The audit log recorded every tool call and result, which is the only reason the next part was possible. Four hundred abandoned tasks, grouped by cause:
| Cause | Tasks | What it looked like |
|---|---|---|
| Customer lookup failed | 141 | The same search repeated with small spelling variations |
| Question spanned two systems | 96 | Correct answer for one system, no tool for the other |
| Ambiguous request | 74 | Two readings; the agent picked one and hedged |
| Tool error with no guidance | 48 | The identical call retried until the step budget ran out |
| Genuinely novel | 38 | Correct handover, working as designed |
Only 38 of 397 were the agent behaving correctly. And two causes — lookup failures and unhelpful tool errors — accounted for 189, neither of which is a fault in the agent.
The lookup tool required an exact match. A transposed character in a company name returned "not found" rather than a suggestion, and the agent's response — try again slightly differently — was reasonable and could never succeed.
The error case was worse. The tool returned error: request failed with no indication of whether a retry might help. The agent retried, unchanged, until its step budget ran out. A tool that cannot say whether an error is retryable makes every agent using it burn its budget.
The four changes#
None of them touched the prompt or the model.
Fuzzy matching in the customer lookup tool. Two days of work in a system nobody thought of as part of the agent. Closed 141 handovers.
Errors that state whether a retry could help. One day. Closed 48 handovers and removed the most wasteful pattern in the log.
Repetition detection. Compare each action to the previous three; if the task is not moving, stop. Failing tasks now stop at around step 5 instead of step 12, which halved the model cost of failures and — more importantly — got a person involved four minutes sooner, with a shorter transcript to read.
One clarifying question. The agent may now ask a single question before choosing between two readings. This addressed the 74 ambiguous cases and had an unexpected effect on the edit rate: drafts that had previously hedged now committed, and needed less rewriting.
The result#
| Before | After | |
|---|---|---|
| Completion rate | 78% | 91% |
| Human time per month | 162 hours | 74 hours |
| Model and tool spend | $840 | $1,110 |
| Loaded cost per resolved ticket | $6.20 | $2.05 |
| Approval gate | Every message | Unchanged |
Model spend went up, because more tasks now run to completion. It remains the smallest term in the calculation and optimising it would still be the wrong place to look.
What was learned#
Count the loaded cost, including human time on failures. Cost per call flatters every agent ever built. The loaded figure is the one that decides whether to continue.
An abandoned task is worse than an untouched one. This is the asymmetry that makes completion rate misleading. The person picking it up pays to understand what already happened before they can start.
Most agent failures are tool failures. Three of the four fixes were in tools the agent called, not in the agent. No amount of prompt work would have found them, and none of them were visible without the transcripts.
Keep the approval gate. It was never questioned during this exercise and it is the reason the 2% tool-failure rate was a cost problem rather than a customer-facing one.