AI Engineering and Product: KPIs
The six measures for shipping learned behaviour, why cost per call is the wrong denominator, and the counter-metrics that catch a feature hitting its numbers by pushing work elsewhere.
Markdown. No sign-up, no email.
Deterministic software is right or wrong against a specification. A learned system is better or worse against a distribution, which means the usual engineering metrics do not transfer. A test suite at 100% green says nothing about whether answers got worse this week.
Every measure below is chosen because it survives that difference.
The six that matter#
| Measure | Definition | Target | How it gets gamed |
|---|---|---|---|
| Eval pass rate, per category | Held-out set results broken down by case type | No category below its floor | Reporting one blended score, which hides the category that collapsed |
| Regressions caught pre-release | Share of quality drops found by the eval rather than by a customer | Above 90% | Shrinking the eval set until nothing fails |
| Cost per successful outcome | Total spend divided by outcomes that actually worked | Falling | Using cost per call instead, which rewards a cheap answer that fails twice |
| Escalation rate to a human | Share of cases the system hands over | Inside a stated band | Suppressing escalations to look autonomous |
| p95 latency | 95th percentile end to end, not the mean | Under the product's stated limit | Quoting the average, which hides the experience people abandon |
| Time to roll back | Measured by rehearsal, not estimated | Under 15 minutes | Never rehearsing, so the number is a guess |
Cost per successful outcome is the one people get wrong. Cost per call makes a cheap model look efficient while it fails a third of the time and the retry, the escalation and the apology all cost more than the better answer would have. Divide by outcomes, not by requests.
The escalation rate is a band, not a target to minimise. Too low means the guardrails are not firing and the system is answering things it should hand over. Too high means the feature is not carrying its weight. Both directions are failures, and only one of them looks like success on a dashboard.
Two counter-metrics#
| Counter-metric | Catches |
|---|---|
| Support contact rate on journeys the feature touches | A feature that hits its latency and cost targets by producing answers people then have to query |
| Human review time per escalated case | Escalation that technically works but hands over cases so poorly framed that reviewing one costs more than doing it from scratch |
What is deliberately not measured#
- Model accuracy in isolation. A model at 94% inside a process that routes 40% of cases to a person anyway has an unclear effect on the outcome. Measure the product.
- Tokens consumed. An input, not a result. It belongs in the cost calculation, not on the scorecard.
- Prompt count or version churn. Activity.
- Benchmark scores. Public benchmarks measure a distribution that is not ours. They inform model selection and they do not belong in operational reporting.
Quality floors, per category#
A single pass rate hides the thing worth knowing. Each category of the held-out set carries its own floor, and a release is blocked if any one of them drops below it, regardless of the blended number.
| Category | Why it gets its own floor |
|---|---|
| Common cases | Volume. A small drop here affects the most people |
| Edge cases | Where quality decays first and quietly |
| Refusals | The system must still decline what it should decline. Refusal quality degrades silently when a model is tuned for helpfulness |
| Adversarial | Prompt injection and jailbreaks. Shared with Security |
Cadence#
| Per release | Eval pass rate by category, cost per outcome, p95 latency |
| Weekly | Drift against the last accepted release, escalation rate, both counter-metrics |
| Quarterly | Rollback rehearsal, eval-set review for contamination and staleness |
The quarterly eval-set review is the one that decides whether any of the rest is trustworthy. An eval set that has leaked into training reports confidence rather than quality, which is worse than having no eval set at all, because a team acts on it.