Workflows · AI Engineering & Product

AI Engineering and Product: Workflows

What arrives, what ships, the eval to release to drift loop, and the handoff contracts with Software Engineering, QA, Security, Support and Legal.

Markdown. No sign-up, no email.

The defining constraint on this function is that its output keeps changing after it ships. A deterministic feature behaves tomorrow as it behaved today. A learned one drifts against a world that moves, so the workflow does not end at release.

What arrives#

FromWhatBecomes
ProductA requirementA feature, once its autonomy boundary is written
SupportComplaints about answersEval cases, which is the only durable use for them
SecurityAdversarial findingsRed-team suite entries and guardrail changes
FinanceSpend by featureThe cost-per-outcome line
Research and InnovationA capability that now worksA candidate, or a note that it still does not fit
Legal and ComplianceDisclosure and marking dutiesRelease-gate items

A complaint becomes an eval case or it is wasted. Answering the individual customer closes a ticket; turning the case into a test is what stops the next hundred.

What ships#

ToWhat
CustomersFeatures, with their disclosure and marking in place
Software EngineeringThe interface contract for the surrounding application
SecurityRed-team results and the guardrail version in force
Knowledge ManagementModel cards and the record of what was tried
FinanceCost per outcome, per feature

The loop#

Why the loop does not end at release Build: Requirement () → Autonomy boundary () → Guardrails (). Prove: Run eval set () → Red team () → Person accepts (). Watch: Release () → Drift watch () → New eval case (). Build Requirement Autonomy boundary Guardrails Prove Run eval set Red team Person accepts Watch Release Drift watch New eval case
A deterministic feature behaves tomorrow as it does today. A learned one drifts, so drift-watch feeds straight back into the eval set.

Handoff contracts#

To Software Engineering. They own the application around the feature; this function owns the behaviour inside it. The contract states what the feature returns, what it does when uncertain, and what it does when the model is unavailable. The third one is skipped most often and is the one that takes the product down.

To QA. Everything deterministic is theirs. Anything measured against a distribution stays here. Mixing the two produces either untested AI features or an engineering team asked to certify something it has no method for.

With Security. Red-team findings are raised here and closed jointly. A prompt injection that works is a security incident, not a quality bug, and routing it as a quality bug loses the severity.

To Support. Before a feature ships, Support gets: what it does, what it refuses, how a customer tells it is AI, and how to escalate. A support team meeting a feature at launch is a support team guessing on a customer's behalf.

With Legal and Compliance. Anything the system asserts to a customer, and the Article 50 disclosure and marking duties that have applied since 2 August 2026.

Cadence#

Per releaseEval by category, red-team pass, named acceptance
WeeklyDrift against last accepted release, escalation rate, cost per outcome
QuarterlyEval-set review, autonomy boundary review, rollback rehearsal

The failure this design is built against#

A feature ships, it works, nobody looks again, and quality decays for a quarter until a customer escalates loudly enough to be heard. Every element here, drift-watch, per-category floors, the complaint-to-eval-case rule and the quarterly boundary review, exists to make that outcome structurally hard rather than merely discouraged.

Back to AI Engineering & Product

Get new material when it is published

Everything here is free and stays free. There is no form in front of any document. If you want to know when new guides and templates go up, leave an email.

Roughly monthly. Unsubscribe in one click. We do not share your address, and we will not call you.