Concept Drills: 16 Load Balancer Probes

Load balancers come up in nearly every system design interview, usually as a follow-up. These are the probes that actually get asked.

Fundamentals

1. Why do we need a load balancer at all? · L4 · Testing: the basics

Three reasons. Scalability — you can add servers and the change is transparent to users, because they only ever know one address. Availability — if a server fails the LB detects it and reroutes to healthy ones. Performance — requests go to servers with the lowest load. Without one, adding servers doesn't add capacity, because nothing routes traffic to them.

2. Where do load balancers go in a three-tier architecture? · L4 · Testing: placement

Between end users and web servers, between web servers and application servers, and between application servers and databases. The general rule is more useful than the three-tier picture though: a load balancer belongs between any two services that have multiple instances, including internal service-to-service calls.

3. What does a load balancer do besides distribute requests? · L5 · Testing: breadth

Health checking via heartbeat, TLS termination to offload crypto from backends, service discovery by querying the registry, predictive analytics on traffic patterns, automated failure handling that cuts manual intervention, and security — mitigating DoS at layers 3, 4, and 7. It's the first thing traffic hits after the firewall, which makes it the natural place for all of it.

4. Isn't the load balancer itself a single point of failure? · L5 · Testing: whether they notice

Yes, and it's in the path of 100% of requests, so its availability has to exceed everything behind it. Minimum is a pair with a floating virtual IP that fails over. At scale you go active-active with ECMP routers distributing across the LB fleet — which is exactly what tier 1 is for.

Global and local

5. What's the difference between GSLB and local load balancing? · L4 · Testing: the two scales

GSLB distributes across geographic regions, choosing a data center based on user location, capacity, and health. Local load balancing distributes within a data center across the server pool. GSLB picks the building; the local LB picks the machine.

6. Why can't DNS handle load balancing on its own? · Staff · Testing: the four specific limits

Four reasons. Packet size — the 512-byte DNS limit means you can't enumerate hundreds of servers. Client behavior — clients may pick randomly from the set and land on a busy target. Proximity — DNS can't easily determine the closest server without geolocation or anycast. Slow recovery — caching and TTL delay failover. The first one is decisive on its own: a 500-server pool simply cannot be expressed in DNS.

7. What are DNS round robin's two failure modes? · L5 · Testing: GSLB limits

Uneven distribution — a large ISP resolver caches one IP and hands it to all its users, so the split is across resolvers rather than users. And no failure detection — round robin keeps handing out a crashed server's IP until the cached entry's TTL expires. Short TTLs mitigate both, which is why providers use them, but they don't eliminate either.

Algorithms

8. Round robin or least connections — how do you choose? · L5 · Testing: matching to workload

By how variable request duration is. If requests are uniform and servers identical, round robin is cheapest and fine. If durations vary a lot — some 5 ms, some 30 seconds — round robin keeps piling work onto a server already tied up, so least connections is right because the in-flight count reflects real occupancy.

9. Static versus dynamic algorithms? · L5 · Testing: the key division

Static algorithms like round robin use fixed configuration: simple, low overhead, blind to server health and load. Dynamic algorithms like least connections monitor current state: more complex, and they require LBs to exchange information, but they route around overloaded and unhealthy servers. In practice dynamic wins, because servers are never really interchangeable — one box is always slower.

10. Ten load balancers all using least connections. What goes wrong? · Staff · Testing: distributed state

Each LB only sees its own connections, so all ten can independently pick the same backend as least loaded and burst it simultaneously. The algorithm creates a herd. Power-of-two-choices fixes it: sample two backends at random and pick the less loaded. Near-optimal balance, no shared state, and two LBs rarely sample the same pair.

11. When would you use IP hash, and what does it cost? · L5 · Testing: stickiness trade-offs

When you need session affinity and can't change the application. It costs you real problems: everyone behind a corporate NAT hashes to one server, mobile users lose sessions when their IP changes, and adding a server reshuffles mappings. I'd much rather make the servers stateless by pushing sessions to a shared cache, and then use an algorithm that actually balances.

State

12. Stateful or stateless load balancing? · Staff · Testing: whether they question the premise

First I'd ask why the LB needs session state at all — if sessions live in a shared cache or a signed client token, the backends are stateless, any server can serve any request, and the question disappears. If I'm stuck with server-local sessions, stateless with consistent hashing, because the real cost of stateful isn't memory, it's synchronizing a session table across every LB on the hot path.

13. What exactly makes a load balancer "stateful"? · Staff · Testing: precision

Synchronization, not storage. If state must be synchronized across multiple load balancers, it's stateful. If it's maintained locally within a single LB or derived algorithmically, it's stateless. So an LB with a local cache of routing decisions is still stateless — nothing has to be replicated for correctness.

14. Why consistent hashing rather than modulo? · L5 · Testing: the scaling property

With hash(client) % N, changing N changes the modulus for everybody, so nearly every client remaps and every session breaks at once. Consistent hashing places servers and keys on a ring, so adding or removing a server only remaps that server's arc — about 1/N of keys. Everything else stays put, which is what makes stateless balancing survive scaling.

Layers and tiers

15. L4 or L7? · L5 · Testing: the core comparison

L4 works at the transport layer on IPs and ports, routes per connection, is content-agnostic, and is faster because there's nothing to parse. L7 works at the application layer, sees URLs, headers, and cookies, routes per request, and can do TLS termination, rate limiting, and header rewriting — at the cost of parsing every request. Need content-based routing, it's L7. Need raw throughput, non-HTTP protocols, or the original client IP preserved, it's L4. At scale you use both, layered.

16. Walk me through the tiered deployment. · Staff · Testing: the full stack

DNS is tier 0, picking the data center. Tier 1 is ECMP routers at layer 3, spreading traffic across the LB fleet by IP hash or round robin — that's what makes the fleet horizontally scalable. Tier 2 is L4 LBs using consistent hashing so every packet of a connection reaches the same tier-3 box; without it you'd get routing errors during scaling and failures. Tier 3 is L7 LBs doing content routing, TLS termination, HTTP health checks, and offloading TCP congestion control and Path MTU discovery so app servers only run business logic. Tier 3 is the widest fleet because it does the most work per request — and it's the most bug-prone for the same reason.

Self-check

The cheat sheet next compresses the chapter onto one page.