AI Engineering and Product: AI Agents
The four agents that test and watch what this function ships, what each may reach, and why the release decision, the guardrail policy and the autonomy boundary never move to a machine.
Markdown. No sign-up, no email.
This is the function most tempted to automate its own judgement, because the work looks measurable. It is not: a number produced by an eval is evidence, and deciding what that evidence justifies is a separate act.
Agents here test, attack, watch and count. They never accept a release.
The four agents#
Evaluation agent#
Runs the held-out set against a candidate change and reports the diff per category.
Reporting by category rather than as one score is the whole design. A blended figure can improve while the edge-case category collapses, and that release then passes on a number nobody questioned.
Reach: the versioned eval set, read-only. It cannot add or remove cases. Output: the diff, the categories that moved, and the ones now below floor.
Red-team agent#
Attacks the system's own guardrails: prompt injection, goal hijacking, tool misuse, and data exfiltration through outputs.
Findings go to Security as well as here, and the agent that raises a finding never closes it. Self-closing findings are how a red-team programme becomes a formality.
Reach: a staging instance only. It never attacks production. Output: reproducible attacks, with the input that worked.
Cost-attribution agent#
Tracks spend per feature and per successful outcome, and flags a feature whose unit economics have inverted.
Per outcome, not per call. A cheap model that fails a third of the time and gets retried is not a saving, and cost per call is the metric that hides it.
Reach: billing and telemetry, read-only. Output: cost per outcome by feature, and the trend.
Drift-watch agent#
Compares live output distributions against the last accepted release and raises a warning when they diverge.
This is the earliest available signal of silent regression, which is the characteristic failure of shipped models. It watches the shape of what the system produces, not whether any single answer was right.
Reach: production output logs, read-only. Output: a divergence warning with the categories affected.
What no agent does here#
- Accept a release. A person reads the diff and signs.
- Change a guardrail. Guardrails are policy, and policy is owned.
- Move the autonomy boundary. Widening what a feature does unsupervised is a decision, recorded, never a side effect of a model upgrade.
- Edit the eval set. An agent that can add cases to the test it is judged by has no value as a judge.
- Close its own finding. Especially the red-team agent's.
The governance position underneath this#
Checked 9 August 2026. The three frameworks most organisations run, the OECD Principles as a values statement, NIST AI RMF as the internal risk model and ISO/IEC 42001 as the certifiable management system, were all written before autonomous agents and none covers them. Extending them by hand for cascading failures, scope creep and attribution gaps is currently the buyer's problem and therefore ours.
Singapore's Model AI Governance Framework for Agentic AI, launched 22 January 2026, is the first written for agents. It holds that an agent cannot be a principal: the organisation stays responsible, a human stays accountable, and delegation is explicit and bounded. It also expects each agent to carry a verifiable identity and an audit trail of which agent acted under whose authorisation.
The list above is that principle applied to this function. Every one of those five exclusions is a point where accountability would otherwise become untraceable, and the test is the same one used everywhere in this framework: not what the agent can do, but who answers when it is wrong.