Checklist · Red Team

AI Red Team Exercise Checklist

Run a structured adversarial exercise, scoping, team composition, the directions to cover, how to record a finding, and turning the output into a regression suite rather than a report.

Download the checklist (Word) Read the full guide

Markdown. No sign-up, no email.

System: _______________ Dates: _______ Lead: _______________

The output should be a regression suite, not a report. A report is read once.

1. Scope: before anyone starts#

[ ] System boundary defined: components, tools, data
[ ] "Bad" defined specifically for this system, not generically
[ ] Worst realistic outcomes written down first
[ ] Test environment or production decided
[ ] Real or synthetic data decided
[ ] Out-of-bounds actions stated
[ ] Who is informed, and when
[ ] Time-boxed. Two focused days beats an open-ended exercise
[ ] Stop-and-report rule agreed for serious findings

2. Team#

[ ] Does not include the people who built it
[ ] Includes a domain expert who knows what would actually be damaging
[ ] Includes someone with a security background
[ ] Includes someone who has never seen the system
[ ] Working in pairs: one drives, one records

3. Directions covered#

[ ] Role and instruction override
[ ] Indirect injection. Hostile content in retrieved material
[ ] Tool misuse, legitimate tool, unintended purpose
[ ] Tool chaining. Two harmless tools combining into harm
[ ] Permission boundary crossing
[ ] Memory / context poisoning
[ ] Overconfidence, led into assertive fabrication
[ ] Domain harm. Advice that is wrong in a way that damages someone
[ ] Cost exhaustion
[ ] Refusal bypass: framing, hypotheticals, role-play, translation
[ ] The boring attacks, just asking directly

4. Method#

[ ] Started from consequences, then worked backwards to technique
[ ] Escalated progressively: misuse → attack → combinations
[ ] Near-misses recorded, not discarded
[ ] Everything timestamped

5. Recording each finding#

[ ] Exact input, verbatim
[ ] Exact output, verbatim
[ ] Consequence. Why it matters, not just that it surprised you
[ ] Reproducibility. Succeeded n times out of m
[ ] Severity based on what an attacker could cause
[ ] Suggested mitigation

6. Afterwards#

[ ] Findings triaged by consequence, not novelty
[ ] Fixed architecturally where possible, most findings are excessive agency
[ ] Every finding converted to a permanent regression case
[ ] Suite re-run to verify each fix
[ ] Re-run scheduled after the next model change
[ ] Next exercise scheduled

Findings summary#

#	Direction	Reproducibility	Consequence	Severity	Owner	Fixed
		/

Sign-off#

	Name	Date
Exercise lead
System owner
Accepted risks approved by

Back to Red Team