Software Architecture: Designing for the Changes You Cannot Predict

Architecture is the set of decisions that are expensive to change later.

That definition does more work than any diagram. Which database you use is architecture. How you name variables is not. Whether services communicate synchronously or through events is architecture. Which testing library you chose is not.

The skill is telling the difference — and spending your limited design effort on the first category.

The decisions that are hard to reverse#

• How the system is split. One deployable unit or many. Almost everything else follows.
• How parts communicate. Direct calls, queues, events. This determines your failure modes.
• Where state lives, and which component owns each piece of data. Data has gravity.
• How you handle failure. Retry, queue, degrade, refuse.
• Your consistency model. Must everything be immediately correct, or is eventual acceptable?
• The trust boundaries. Where authentication and authorisation happen.

Everything else — frameworks, libraries, code organisation — is genuinely reversible. Teams routinely spend weeks on framework selection and minutes on the consistency model, which is exactly backwards.

Start with one thing#

The most useful default for a new system: one deployable application, with clear internal boundaries.

This is unfashionable and it is usually right. A single application is easier to develop, test, deploy, debug and reason about. You get transactions for free, no network between components, and a single place to look when something breaks.

The internal boundaries matter, though. Organise by business capability rather than technical layer — billing, ordering, notifications, not controllers, services, repositories. Capability boundaries are where you would eventually split; layer boundaries run across every capability and split nothing usefully.

Do that, and if you later need separate services, the seams already exist. Skip it, and splitting means untangling first.

When splitting genuinely helps#

Not "when the system gets big". Split when you have a specific problem that separation solves:

• Different scaling needs. One component needs ten times the capacity of the rest.
• Different release cadence. One part must ship daily while another is change-controlled.
• Team autonomy. Multiple teams blocking each other on one deployment pipeline.
• Genuine isolation requirements. Regulatory or security boundaries.
• Different technology genuinely required. Rare, and usually claimed more than it is true.

What you get in exchange: network calls that fail, distributed transactions you cannot have, harder debugging, more deployment machinery, and eventual consistency to reason about. That is a real price, and it is worth paying only against a real problem.

"It will be easier to scale later" is not a reason. You can split later. You cannot un-split easily.

Synchronous or asynchronous#

The communication choice determines your failure behaviour more than any other decision.

Synchronous (a direct request, waiting for a reply) is simple to understand and debug. Its weakness is coupling: if the thing you call is down or slow, you are down or slow. Chains of synchronous calls multiply this — five services at 99.9% availability, called in sequence, give you 99.5%.

Asynchronous (a message onto a queue, processed later) survives the other side being unavailable, absorbs load spikes, and decouples release cycles. Its cost is complexity: work completes later, you need to handle duplicates and ordering, and tracing a request across the system becomes real work.

A practical rule: synchronous when the caller genuinely needs the answer to proceed; asynchronous when it does not. Sending a confirmation email does not need to block a purchase.

Designing for failure#

Everything fails. Architecture is largely deciding what happens when it does.

Timeouts on everything. A call without a timeout waits forever, holding resources, until the whole system stalls. This is the single most common cause of cascading failure.

Retry with backoff, and only for transient errors. Retrying a validation failure just fails repeatedly; retrying immediately in a loop turns a brief outage into an outage you caused.

Make operations idempotent. If a retry might duplicate a payment, the design is wrong. This is what makes retries safe at all.

Degrade rather than collapse. If recommendations are unavailable, show the page without them. Decide in advance which features are optional.

Isolate failures. One slow dependency should not consume every connection and take down unrelated functionality.

Recording decisions#

The highest-value architecture artefact is not a diagram. It is a short record of each significant decision: what was decided, what was rejected, and why.

Six months later the rejected options are the valuable part. They stop someone re-litigating a settled question, and they stop someone repeating an approach that was already ruled out for reasons nobody remembers.

Keep them short — a page — in version control, next to the code. Our Architecture Diagram template includes a decisions section.

Diagrams that help#

Most architecture diagrams are unreadable because they try to show everything at once. Draw a small number at decreasing zoom: the system and who uses it, then the deployable parts, then the inside of one part if it warrants it.

The rule that fixes most diagrams: label every arrow. An unlabelled arrow is the most common defect — nobody can tell whether it means "calls", "sends data to", or "occasionally reads from", and those have completely different implications.

FAQ#

Microservices or a monolith?#

Start with one well-structured application. Split when a specific problem demands it. Most teams that adopted microservices early report the distributed complexity cost more than the coupling they were avoiding.

How do we know if our architecture is wrong?#

Symptoms are practical, not aesthetic: a simple change requires touching many components; deploying one thing means coordinating several teams; nobody can explain what happens when a given component fails; the same data is authoritative in two places.

Who should be the architect?#

Ideally someone who still writes code in the system. Architecture decided by people insulated from the consequences drifts toward the theoretically elegant and practically painful.

How much design before building?#

Enough to settle the hard-to-reverse decisions listed above. Not more. Detailed design of things you have not built yet is usually wrong in ways you cannot discover until you build them.

What about performance?#

Design so you can measure it, then measure before optimising. Most performance problems in business systems are database access patterns, not language or framework choices — and they are found by profiling, not by guessing.

Should we build for scale we do not have?#

Build so scaling is possible — stateless components, no assumption of a single instance, data access you can optimise. Do not build the scaled architecture itself. You will optimise for the wrong dimension and pay for it daily.

How do we handle a system we inherited?#

Map what exists before changing anything, especially data ownership and the failure modes. Then improve incrementally where you already work. Wholesale rewrites of systems nobody fully understands have a poor record.

What else is coming for Architecture