# AI Red Team Exercise Checklist

**System:** _______________  **Dates:** _______  **Lead:** _______________

> The output should be a **regression suite**, not a report. A report is read once.

## 1. Scope — before anyone starts

- [ ] System boundary defined: components, tools, data
- [ ] **"Bad" defined specifically for this system**, not generically
- [ ] Worst realistic outcomes written down first
- [ ] Test environment or production decided
- [ ] Real or synthetic data decided
- [ ] Out-of-bounds actions stated
- [ ] Who is informed, and when
- [ ] **Time-boxed** — two focused days beats an open-ended exercise
- [ ] Stop-and-report rule agreed for serious findings

## 2. Team

- [ ] **Does not include the people who built it**
- [ ] Includes a domain expert who knows what would actually be damaging
- [ ] Includes someone with a security background
- [ ] Includes someone who has never seen the system
- [ ] Working in pairs: one drives, one records

## 3. Directions covered

- [ ] Role and instruction override
- [ ] **Indirect injection** — hostile content in retrieved material
- [ ] Tool misuse — legitimate tool, unintended purpose
- [ ] **Tool chaining** — two harmless tools combining into harm
- [ ] Permission boundary crossing
- [ ] Memory / context poisoning
- [ ] Overconfidence — led into assertive fabrication
- [ ] **Domain harm** — advice that is wrong in a way that damages someone
- [ ] Cost exhaustion
- [ ] Refusal bypass: framing, hypotheticals, role-play, translation
- [ ] The boring attacks — just asking directly

## 4. Method

- [ ] Started from **consequences**, then worked backwards to technique
- [ ] Escalated progressively: misuse → attack → combinations
- [ ] Near-misses recorded, not discarded
- [ ] Everything timestamped

## 5. Recording each finding

- [ ] Exact input, verbatim
- [ ] Exact output, verbatim
- [ ] **Consequence** — why it matters, not just that it surprised you
- [ ] **Reproducibility** — succeeded n times out of m
- [ ] Severity based on what an attacker could cause
- [ ] Suggested mitigation

## 6. Afterwards

- [ ] Findings triaged **by consequence, not novelty**
- [ ] Fixed architecturally where possible — most findings are excessive agency
- [ ] **Every finding converted to a permanent regression case**
- [ ] Suite re-run to verify each fix
- [ ] Re-run scheduled after the next model change
- [ ] Next exercise scheduled

## Findings summary

| # | Direction | Reproducibility | Consequence | Severity | Owner | Fixed |
|---|---|---|---|---|---|---|
| | | /  | | | | |

## Sign-off

| | Name | Date |
|---|---|---|
| Exercise lead | | |
| System owner | | |
| Accepted risks approved by | | |