Engineering: KPIs
Six delivery measures for a pipeline where code is cheap, why lines and velocity are excluded, and the gate-effectiveness numbers that show whether the controls are real.
Markdown. No sign-up, no email.
When generation costs almost nothing, every measure of output volume becomes noise. More code is not an achievement; it is usually a liability with a longer maintenance tail.
These measure flow, quality and whether the gates are doing anything.
The six that matter#
| Measure | Definition | Target | How it gets gamed |
|---|---|---|---|
| Change lead time | First commit to running in production | Under 2 days | Starting the clock at the pull request rather than the first commit |
| Deployment frequency | Successful production deploys | Daily or better | Batching, then counting the batch as one healthy deploy |
| Change failure rate | Deploys causing a rollback or hotfix | Under 15% | Calling a hotfix a feature |
| Time to restore | Detection to service restored | Under 1 hour | Measuring from when someone opened a ticket rather than when it broke |
| Escaped defect rate | Defects found in production per released change | Falling | Reclassifying production defects as enhancements |
| Gate effectiveness | Share of defects stopped by a gate rather than by a customer | Above 90% | Loosening the gates, which improves the ratio by letting less get caught anywhere |
The first four are the DORA set and they are used here because they resist the volume problem: none of them improves by writing more code.
Gate effectiveness is the one specific to this model. A pipeline of six automated gates either stops things or it is decoration, and the only way to tell is to count what each one caught against what reached a customer.
Gate-level numbers#
Reviewed monthly, per gate.
| Per gate | Why |
|---|---|
| Times fired | Zero over a month is an investigation, not a success. Either it is misconfigured or work is routing around it |
| Times bypassed, and by whom | A bypass with no recorded reason is a process failure regardless of outcome |
| False positive rate | A gate people learn to ignore is worse than no gate: it produces the appearance of control |
| Defects it caught that no later gate would have | This is what justifies the gate existing at all |
Two counter-metrics#
| Counter-metric | Catches |
|---|---|
| Review depth on AI-generated changes | Volume rising while human attention per change falls. The failure mode of cheap generation is not bad code, it is more code than anyone reads |
| Time spent on unplanned work | A team hitting its delivery numbers by continuously repairing what it shipped last week |
What is deliberately not measured#
- Lines of code. Now actively misleading. Generation makes it trivial to grow and it correlates with maintenance cost, not value.
- Commits or pull requests. Activity.
- Velocity or story points. Estimates measured against themselves. They inflate quietly and nobody can say when it happened.
- AI suggestion acceptance rate. Measures how agreeable the tool is, not whether the system improved. It is the vanity metric of this whole category.
Cadence#
| Weekly | Lead time, deploy frequency, change failure rate, unplanned work |
| Monthly | Gate audit, escaped defects, review depth |
| Quarterly | Technical debt review, and whether the targets above are still the right ones |
The quarterly question is the one usually missing. A target that has been hit for four quarters is either genuinely solved or no longer measuring anything, and those two look identical until someone asks.