Checklist · Red Team

AI Red Team Exercise Checklist

Run a structured adversarial exercise — scoping, team composition, the directions to cover, how to record a finding, and turning the output into a regression suite rather than a report.

Download the checklist Read the full guide

Markdown. No sign-up, no email.

System: _______________ Dates: _______ Lead: _______________

The output should be a regression suite, not a report. A report is read once.

1. Scope — before anyone starts#

[ ] System boundary defined: components, tools, data
[ ] "Bad" defined specifically for this system, not generically
[ ] Worst realistic outcomes written down first
[ ] Test environment or production decided
[ ] Real or synthetic data decided
[ ] Out-of-bounds actions stated
[ ] Who is informed, and when
[ ] Time-boxed — two focused days beats an open-ended exercise
[ ] Stop-and-report rule agreed for serious findings

2. Team#

[ ] Does not include the people who built it
[ ] Includes a domain expert who knows what would actually be damaging
[ ] Includes someone with a security background
[ ] Includes someone who has never seen the system
[ ] Working in pairs: one drives, one records

3. Directions covered#

[ ] Role and instruction override
[ ] Indirect injection — hostile content in retrieved material
[ ] Tool misuse — legitimate tool, unintended purpose
[ ] Tool chaining — two harmless tools combining into harm
[ ] Permission boundary crossing
[ ] Memory / context poisoning
[ ] Overconfidence — led into assertive fabrication
[ ] Domain harm — advice that is wrong in a way that damages someone
[ ] Cost exhaustion
[ ] Refusal bypass: framing, hypotheticals, role-play, translation
[ ] The boring attacks — just asking directly

4. Method#

[ ] Started from consequences, then worked backwards to technique
[ ] Escalated progressively: misuse → attack → combinations
[ ] Near-misses recorded, not discarded
[ ] Everything timestamped

5. Recording each finding#

[ ] Exact input, verbatim
[ ] Exact output, verbatim
[ ] Consequence — why it matters, not just that it surprised you
[ ] Reproducibility — succeeded n times out of m
[ ] Severity based on what an attacker could cause
[ ] Suggested mitigation

6. Afterwards#

[ ] Findings triaged by consequence, not novelty
[ ] Fixed architecturally where possible — most findings are excessive agency
[ ] Every finding converted to a permanent regression case
[ ] Suite re-run to verify each fix
[ ] Re-run scheduled after the next model change
[ ] Next exercise scheduled

Findings summary#

#	Direction	Reproducibility	Consequence	Severity	Owner	Fixed
		/

Sign-off#

	Name	Date
Exercise lead
System owner
Accepted risks approved by

Back to Red Team