Research and Innovation: SOPs
Five procedures covering question intake, experiment design with kill criteria, replication, the negative-results record and the handover to product.
Markdown. No sign-up, no email.
SOP 1: Question intake#
Run: on every proposed question.
| Test | Fails if |
|---|---|
| Would the answer change a decision? | Nobody can name the decision |
| Is the decision ours to make? | It belongs to a customer or a vendor |
| Has it been answered before? | The prior-work record already contains it |
| Can it be settled in six weeks? | If not, decompose it or decline it |
The first test rejects most of the queue, and that is what makes the rest of the function affordable. Interesting is not the bar; decision-changing is.
SOP 2: Experiment design#
Run: before any work starts.
- The cheapest test that could plausibly settle the question. Not the most rigorous, the cheapest sufficient one.
- The kill criterion, written and agreed in advance. What result would make us stop.
- A time box, with a date it ends regardless of progress.
- What we predict. Recorded before the run, because a prediction written afterwards always matches.
Kill criteria written afterwards are not kill criteria. They become descriptions of whatever happened, and the experiment can then never fail.
SOP 3: Replication#
Run: on every positive result, before it leaves the function.
Re-run independently, by someone who did not run it the first time.
Most reversals happen here. That is not a defect of the process, it is the process working: a positive result that survives replication is worth acting on, and one that does not would have cost far more downstream.
SOP 4: The negative-results record#
Run: on every killed experiment, before it is closed.
| Field | Why |
|---|---|
| The question | So it is findable when asked again |
| What we tried | So the next attempt is different |
| What it cost | So the value of not repeating it is visible |
| Why it failed | Distinguishing "does not work" from "did not work for us, this way, then" |
| What would have to change | The condition for revisiting. Without it, a no becomes permanent by accident |
An experiment closed without a write-up has been paid for and not banked. This is the single most valuable procedure in this function and the easiest to skip.
SOP 5: Handover to product#
Run: when a finding is ready to become something.
- Replicated.
- Written up so someone who was not there can act on it.
- Handed to AI Engineering and Product or to AI Strategy, who decide whether it becomes work.
- The research team does not productise its own finding. The team that proved something works is the worst judge of whether it is ready.
Escalation#
| Situation | Goes to |
|---|---|
| An experiment past its time box | Kill it, or a named person extends it in writing |
| A result that changes strategy | CEO and AI Strategy, immediately |
| A finding with commercial value | Product, with Legal consulted on anything protectable |
| Nobody outside the function is asking questions | CEO. The queue has drifted from what the company needs |