Load balancing

Problem

What becomes confusing, fragile, or impossible without understanding load balancing? This lesson answers that through explanation, a worked example, two runnable exercises, and a reference solution. No teacher-supplied worksheet is required.

Distributed architectures address concrete capacity, reliability, and coordination limits—but often arrive before those limits do.

Concept explanation

Load balancing belongs to “Distributed systems without cargo cult”. load balancing must be connected to layer below by tracing representation, ownership, control, and failure. Replication, partitioning, consistency, balancing, and observability trade coordination for capacity/availability.

For load balancing, trace concrete input, state transition, output, and failure through an architecture decision tied to explicit requirements, estimates, state ownership, failure modes, and measured limits.

Start with requirements and estimates, choose the smallest architecture, locate its first bottleneck, then evolve one measured constraint at a time. Apply that model to supplied normal, boundary, and failure cases; each case below names its input and expected evidence.

Example bank

Compare normal, boundary, failure, and cross-layer cases. Predict each observation before revealing the explanation.

Worked example

Start from supplied design.md. Focus: Trace load balancing end-to-end while running: Design chat/search from single node, then add one measured distributed constraint at a time.

1. Run: Fill design.md, then verify every component traces to one stated requirement or measured constraint.
2. Save baseline evidence. Trace requests, state ownership, capacity, queues, caches, replication, failure domains, recovery, operational load, and cost.
3. Boundary case: At every load balancing boundary, label owner and representation during: Model partition and replica lag in user-visible terms.
4. Failure case: Locate first layer where evidence diverges during: Lose leader or duplicate delivery and specify recovery/invariant.

RESULT
Trace explains Diagram names state owner, consistency window, failure domain, trace path, and coordination tax. without skipping any conversion, queue, process, protocol, or storage boundary. Starter-level baseline: Document names one action, target, non-goal, numeric estimate, state owner, request path, bottleneck, and recovery procedure.

Starter material

Create design.md, paste this exact content, then run the command below.

# load balancing

## Requirements
- One primary user action:
- One reliability target:
- One explicit non-goal:

## Estimate
- Requests/second:
- Stored bytes/day:
- Peak concurrent work:

## Smallest design
- State owner:
- Request path:
- First bottleneck:
- Recovery procedure:

STOP / CLEANUPStop any process started by your slice with Ctrl+C; run git status before leaving.

Independent exercise

Remove one abstraction: load balancing

1. Create second case from blank file: At every load balancing boundary, label owner and representation during: Model partition and replica lag in user-visible terms.
2. Then create controlled failure: Locate first layer where evidence diverges during: Lose leader or duplicate delivery and specify recovery/invariant.
3. Use Architecture notebook · load tests · traces · Git history to prove behavior, then repair controlled failure.
4. Compare result against supplied acceptance checks and reference approach before marking complete.

Expected result

• Trace explains Diagram names state owner, consistency window, failure domain, trace path, and coordination tax. without skipping any conversion, queue, process, protocol, or storage boundary.
• Document names one action, target, non-goal, numeric estimate, state owner, request path, bottleneck, and recovery procedure.
• Controlled load balancing failure produces captured evidence; repair restores stated invariant without hiding error.

Hints

Solution

Attempt both exercises before opening reference approach.