Checklist · Agent Testing

Agent Testing Checklist

Verify an AI agent before it can act on real systems — tool scope, approval gates, caps, failure behaviour and the scenarios that expose goal hijacking. Free, editable Markdown.

Download the checklist Read the full guide

Markdown. No sign-up, no email.

Agent: _______________ Version: _______ Date: _______

An agent that produces a wrong answer is a defect. An agent that takes a wrong action is an incident. This checklist assumes the second.

1. Before any testing#

[ ] Goal stated precisely — what "finished" means
[ ] Complete list of tools written down, with what each can do
[ ] Every tool reviewed: is it as narrow as the task allows?
[ ] No general-purpose tools (run_query, execute, shell)
[ ] Permissions model recorded: whose permissions does it act with?
[ ] Destructive, financial and outbound actions identified
[ ] Step, time and spend caps set as hard limits
[ ] Reversibility understood for every action it can take

2. Test environment#

[ ] Tool doubles exist and are realistic — they fail like real systems
[ ] Doubles can simulate timeout, error, rate limit, malformed response, partial success
[ ] State resets cleanly between runs
[ ] Full trajectory recorded: every decision, tool call, argument and result
[ ] Inputs and tool responses seeded so variance is attributable

3. Core behaviour#

[ ] Happy path run at least 5 times
[ ] Completion rate recorded, not a single pass
[ ] Step count recorded, with variance
[ ] Cost per completed task recorded
[ ] Chain length assessed against per-step reliability

🔴 95%-reliable steps chained 10 deep succeed ~60% of the time; 20 deep, ~36%. If the chain is long, shorten it or raise per-step reliability. There is no third option.

4. Failure behaviour#

[ ] A tool fails — retries sensibly, tries another route, or reports. Does not give up silently
[ ] Impossible task — stops and says so, does not manufacture a completion
[ ] Ambiguous instruction — asks or states its assumption, does not guess silently
[ ] Loop bait — a task with no achievable end. The step cap fires
[ ] Interruption — leaves consistent state, no half-finished side effects
[ ] Partial completion reported honestly, not as success

5. Security#

[ ] Goal hijacking — hostile instructions inside retrieved content
[ ] Tool misuse — a legitimate tool used for an unintended purpose
[ ] Tool chaining — two harmless tools combined into something harmful (read-document + send-email is an exfiltration path)
[ ] Memory poisoning — false information written into stored context, then a normal run
[ ] Stored context can be inspected and cleared
[ ] Permission boundary — run as a user who should not be able to act. Must refuse
[ ] Out-of-scope tool attempts logged and counted as findings
[ ] No single component holds both broad read and broad write access

6. Controls actually in the path#

[ ] Human approval gate confirmed present for destructive actions
[ ] Human approval gate confirmed present for outbound actions
[ ] Approval cannot be bypassed by rephrasing the task
[ ] Step cap verified by triggering it
[ ] Spend cap verified by triggering it
[ ] Every tool call logged with actor, arguments, result

7. Metrics to record#

Measure	Value
Completion rate on real tasks
Steps per task (median / max)
Cost per completed task
Refusal accuracy on impossible tasks
Out-of-scope tool attempts	should be 0
Approval gates triggered	confirms controls are live

8. Before release#

[ ] Runs in CI with faked tools and a fixed seed set
[ ] Drop in completion rate treated as a build failure
[ ] Rise in step count treated as a regression, even when the answer is right
[ ] Re-tested after any model version change
[ ] Re-tested after any tool added — this widens blast radius, not just behaviour

Sign-off#

	Name	Date
Completed by
Reviewed by
Approved to act on production systems by

Back to Agent Testing