Failure Isolation and Blast Radius Diagram
A request drawn against what fails with it — where a synchronous call propagates failure, where a boundary contains it, and the four mechanisms that decide blast radius.
SVG. No sign-up, no email.
Architecture diagrams usually show what calls what. That is useful for understanding the system and useless for the question that matters at 3am: when this breaks, what else breaks?
The diagram below is the same system drawn against failure. Each boundary either contains a failure or passes it on, and which one it does is a choice — usually an unexamined one.
Reading the request path#
Every arrow in the top lane is a coupling, and a synchronous call is the strongest kind: the caller cannot complete until the callee does, so the caller's availability can never exceed the callee's. Chain four of those and the arithmetic is unforgiving — four dependencies at 99.9% each give the caller at best 99.6%, before anything of its own goes wrong.
The two shaded boxes are where failure propagates. Pricing is called synchronously, so pricing being slow makes ordering slow, and pricing being down makes ordering down. The shared database is worse: it couples services that have no other relationship, so a query from reporting can degrade order capture, and neither team will look at the other during the incident.
The four mechanisms that contain failure#
Drawn as a set because they are independent and often combined.
A timeout with a default converts an unbounded wait into a bounded one with a known outcome. The default is the design question: if pricing does not answer in 200 milliseconds, is there an acceptable answer — a cached price, a standard rate, a refusal? Sometimes there is not, and then the coupling is genuine and should be stated as such.
A queue changes the question from "can pricing answer now" to "will pricing answer eventually". It is the strongest form of containment and it is not free: the caller must accept that the work happens later, which changes what can be shown to the user and adds a state that did not exist before.
A cache with a maximum age serves the last known good answer. The maximum age is what makes it safe. Without one, a dependency that has been down for six days is silently answered from six-day-old data, and the system reports success throughout.
A circuit breaker stops calling something that is failing. Its value is not to the caller — it is that a struggling service is not held down by the retry traffic of everyone waiting for it. Without one, a slow dependency and its callers form a loop that prevents recovery.
The four questions#
The bottom lane is the review, applied to every arrow in a design.
Slow or down? Down is easy: it fails immediately and something notices. Slow is the case that takes systems out, because everything upstream waits, connection pools fill, and the failure spreads to requests that never needed the slow dependency at all.
Who notices? A failure a user sees is honest. A failure that lands in a queue nobody watches is discovered a week later.
What is lost? The request, or the data? A lost request can be retried by a person. Lost data cannot be recovered by anyone.
How does it recover? Automatically when the dependency returns, or does someone have to do something? Recovery requiring manual intervention is a design decision, and it should be one somebody made.
Using this diagram#
Draw your own system's request path and mark every arrow with what happens when the far end is slow. Two results are common and both are informative.
Most arrows have no answer — nobody decided, so the behaviour is whatever the HTTP client library defaults to, which is usually an unbounded wait.
And the arrows that do have answers are usually protected against the dependency being down rather than being slow, which is the less common and less damaging of the two.