AI Operations
Monitoring on the metrics that predict AI failure rather than the ones that are easy to chart: task success, intervention rate and cost per completed task.
| How it is bought | Monthly recurring |
| Duration | 12-month term, 30 days notice after month 3 |
| Written for | Organisations running AI in production who have discovered that shipping it was the easy part |
An AI system in production degrades quietly. Inputs change shape, the provider updates a model, cost per task creeps, and the first anyone notices is a complaint rather than a metric.
What you receive
Named artifacts, not activities. If one of these is not delivered, the engagement is not complete.
- Monitoring on the metrics that predict failure: task success against the evaluation set, human intervention rate, and cost per completed task
- Evaluation set maintenance, because a test set that never changes stops testing anything
- Model and prompt version control, with a tested rollback
- Drift review: what changed in the inputs, and what it did to quality
- A monthly report against the baseline captured before launch
How it runs
- Month 1: instrument what exists, establish or recover the baseline, and agree the alert thresholds.
- Monthly: quality review against the evaluation set, cost review, and any tuning the drift requires.
What this deliberately does not include
Published with the same prominence as the deliverables. A scope with no stated exclusions is a scope that will be argued about later, and the argument always happens at the worst possible moment.
- Building new AI systems. This operates what exists; new work is a project.
- Model training or fine-tuning unless separately agreed.
- Responsibility for a system we cannot measure. If there is no evaluation set we will build one first, as a project.
How you know it is finished
You can state this month's task success rate, intervention rate and cost per task, and see the trend against launch.
What you need ready
Access to the running system, its logs, and whatever evaluation set exists.
Every engagement starts with a written scope confirmation before any invoice is raised. Nothing here is a click-to-buy: we confirm what you need, agree the scope in writing, then invoice. If the scoping conversation shows that a smaller engagement, or none at all, is the right answer, we say so before anything is signed.
Request this engagement Talk it through first Check your true unit cost
We use privacy-friendly analytics by default. Optional analytics cookies load only if you accept.