Cheat Sheet
Key takeaway
A load balancer distributes requests across a server pool so no single server is overwhelmed. It is what makes horizontal scaling work, and it lives behind one stable address so the fleet can change without clients noticing.
Key terms
| Term | One line |
|---|---|
| Load balancer (LB) | Distributes client requests across a pool of servers |
| GSLB | Global server load balancing — picks the data center |
| Local LB | Picks the server inside a data center; a reverse proxy |
| VIP | Virtual IP — the stable address fronting the pool |
| Reverse proxy | Acts on behalf of servers; clients never see the pool |
| ECMP | Equal cost multipath routers — tier-1 LBs at layer 3 |
| Consistent hashing | Ring-based mapping; adding a node remaps only ~1/N of keys |
| Draining | Stop new connections, let in-flight ones finish |
| Slow start | Ramp traffic to a newly added server gradually |
| DSR / DR | Direct server return — response bypasses lower tiers |
| LBaaS | Load balancer as a service (cloud-managed) |
Why you need one
| Benefit | What it gives you |
|---|---|
| Scalability | Add servers transparently to end users |
| Availability | Detects faults, reroutes to healthy servers |
| Performance | Sends requests to the least-loaded server |
Placement: users → web · web → app · app → database. General rule: between any two services with multiple instances, including internal ones.
Services offered: health checking (heartbeat) · TLS termination · predictive analytics · reduced human intervention · service discovery · security (DoS at L3/L4/L7).
Global vs local
| GSLB | Local | |
|---|---|---|
| Scope | Across regions | Within a data center |
| Decides on | User location, capacity, DC health | Server load, health, content |
| Usually implemented via | DNS, ADCs, cloud LBaaS | Reverse proxy behind a VIP |
Local LBs report health up a control plane to GSLB, which routes on live capacity.
Why DNS alone can't do local balancing
1. Packet size -- 512-byte limit; can't list hundreds of servers 2. Client behavior -- clients may pick randomly, hit a busy target 3. Proximity -- can't find nearest without geolocation/anycast 4. Slow recovery -- caching + TTL delay failover
DNS round robin's two failure modes: uneven distribution (one ISP resolver caches one IP for thousands of users) · no failure detection (keeps serving a dead IP until TTL expires). Mitigate with short TTLs.
Algorithms
| Algorithm | Decides by | Use when |
|---|---|---|
| Round robin | Sequential loop | Identical servers, uniform requests |
| Weighted round robin | Assigned weights | Servers have different capacities |
| Least connections | Fewest active connections | Request durations vary a lot |
| Least response time | Lowest active response time | Latency is the priority |
| IP hash | Hash of client IP | Need stickiness, can't change the app |
| URL hash | Hash of the URL | Distinct clusters per workload |
Also: randomized, weighted least connections, power of two choices.
| Static | Dynamic | |
|---|---|---|
| Example | Round robin | Least connections |
| Basis | Fixed config | Live server state |
| Overhead | Low | Higher (LBs must exchange info) |
| Slow server | Keeps feeding it | Routes around it |
Dynamic wins in practice — servers are never truly interchangeable.
Stateful vs stateless
Stateful = state is synchronized across multiple LBs. Stateless = state is local to one LB or derived algorithmically.
| Stateful | Stateless | |
|---|---|---|
| Routing decided by | Session table lookup | Hash of the key |
| Shared across LBs | Must be synchronized | Nothing to share |
| Weight | Heavier | Faster, lighter |
| Scaling the LB tier | Hard | Trivial |
| Scaling the server pool | Graceful | Less graceful (remapping) |
Consistent hashing, not modulo: hash % N remaps everyone when N changes; ring-based remaps only ~1/N.
Best answer: push session state to a shared cache so backends are stateless and affinity isn't needed at all.
L4 vs L7
| L4 | L7 | |
|---|---|---|
| Layer | Transport (TCP/UDP) | Application (HTTP) |
| Sees | IPs, ports | URLs, headers, cookies |
| Granularity | Per connection | Per request |
| Speed | Faster (no parsing) | Slower |
| Content routing | ✗ | ✓ |
| TLS termination | Some | ✓ |
| Rate limit, header rewrite | ✗ | ✓ |
L4 moves connections; L7 understands requests. Terminating TLS means re-encrypting to the backend if plaintext inside the perimeter isn't acceptable.
Tiered deployment
Tier 0 DNS -> picks the data center
Tier 1 ECMP routers -> layer 3; balances load AMONG load balancers
Tier 2 L4 LBs -> layer 4; consistent hashing pins a connection to one tier-3 LB
Tier 3 L7 LBs -> layer 7; content routing, TLS, HTTP health checks
offloads TCP congestion control, Path MTU discovery
Summary: tier 1 balances load among LBs · tier 2 ensures smooth transitions and consistency · tier 3 does application balancing and network offload.
Tier 3 is the widest fleet (most work per request) and the most bug-prone (most complexity).
DSR / direct server return: response goes back through tier 3 to the routers, skipping tier 2 — responses are much larger than requests. Does not work with TLS termination, since the response must be encrypted at the L7 LB that holds the keys and connection state.
Implementation
| Hardware | Software | Cloud (LBaaS) | |
|---|---|---|---|
| Cost | Expensive | Cost-effective | Metered |
| Flexibility | Vendor lock-in | Programmable | Config-driven |
| HA | Costly (redundant hardware) | Cheap (shadow LBs) | Built in |
| Config | Difficult | Straightforward | Easiest |
| Strength | Peak per-box performance | Scales, predictive analysis | GSLB, auditing, monitoring |
Client-side LB: the client picks the instance from a registry directly — no extra hop, but balancing logic in every client. The modern compromise is a service mesh sidecar.
Quick decision cues
- Adding servers must be invisible to clients → LB behind a stable VIP
- Request durations vary → least connections, not round robin
- Many LBs, large pool → power of two choices
- Need path or header routing → L7
- Need max throughput or non-HTTP → L4
- Need session affinity → push state to a shared cache first; IP hash only as fallback
- LB fleet outgrew one VIP → ECMP tier 1 in front
- Removing a backend → drain, don't delete
- Adding a backend → slow start, or least-connections will flood it
- Control plane unreachable → stay statically stable on last known good config
Work the Interview Walkthrough for the full design and the Concept Drills for rapid-fire practice.