Local Load Balancing
Why this matters: GSLB gets a user to the right building. Everything about which machine inside that building serves the request — and how fast a broken one is taken out of rotation — is the local layer's job.
Key takeaway
Local load balancers reside within a data center and act as a reverse proxy, distributing incoming requests among a pool of available servers. Clients connect to the LB through a virtual IP address (VIP), which abstracts away the backend infrastructure entirely.
Why DNS is not enough
DNS aids global distribution but is insufficient for granular control, for four specific reasons:
| Limitation | The detail | Consequence |
|---|---|---|
| Packet size | The 512-byte limit on DNS packets prevents sending a comprehensive list of all server IP addresses | You physically cannot enumerate a fleet of hundreds of servers over DNS |
| Client behavior | Clients may randomly select an IP from the set they receive | A client can pick a busy data center or server despite better options being listed |
| Proximity | DNS cannot easily determine the closest address without complex geolocation or anycast solutions | Routing is approximate at best |
| Slow recovery | Caching and long TTL values delay recovery during failures | Traffic keeps flowing to a dead target until caches expire |
Read the first row carefully — it is the most concrete of the four. A response that must fit in 512 bytes holds only a handful of addresses. A data center with 500 servers cannot be represented in DNS at all, which settles the question of whether DNS could ever be the only balancing layer.
To address all four, a layer of local load balancing is required.
What a local load balancer is
Two properties define it:
It is a reverse proxy. A forward proxy acts on behalf of clients; a reverse proxy acts on behalf of servers. Clients believe they are talking to the service itself and never learn that a pool exists behind it.
It publishes a virtual IP. The VIP is a single stable address that fronts the whole pool. Servers can be added, removed, replaced, or rebooted with no client-visible change — which is the property that makes the fleet elastic.
| Concern | DNS-only | With a local LB |
|---|---|---|
| Pool size limit | A few addresses (512-byte packet) | Unbounded — the LB knows the full pool |
| Failure detection | TTL-bounded, minutes | Health checks, seconds |
| Server choice | The client picks, possibly at random | The LB picks, using live state |
| Adding a server | Edit a record, wait for propagation | Register it; effective immediately |
| Visibility into load | None | Connections, response times, error rates |
Health checking is what makes it trustworthy
A load balancer that routes to dead servers is worse than none, so health checking is the core competency. Two complementary mechanisms:
| Type | How it works | Strength | Weakness |
|---|---|---|---|
| Active health checks | LB periodically probes each server on a health endpoint | Detects a server that is down even when idle | Costs probe traffic; a shallow probe misses real breakage |
| Passive health checks | LB observes real request outcomes — errors, timeouts | Free, and reflects what users actually experience | Needs traffic to notice; can't tell if a recovered server is ready |
Production systems run both: active checks decide pool membership, passive observation catches the degradations a probe would pass.
Two behaviors worth designing in
| Behavior | What it does | What breaks without it |
|---|---|---|
| Connection draining | On removal, stop sending new requests but let in-flight ones finish | Every deploy and scale-down kills live user requests |
| Slow start / warm-up | Ramp traffic to a newly added server gradually | A cold server with empty caches and an unwarmed JIT gets full load and immediately looks unhealthy |
Key takeaway
GSLB picks the region; the local load balancer picks the server. It can do so because it has what DNS lacks — the full pool, live health, and immediate effect — behind one stable virtual address.
Interview signal by level
| Level | What a strong answer sounds like |
|---|---|
| L4 | "A load balancer inside the data center spreads requests over the servers." |
| L5 | Justifies the layer: "DNS can't enumerate hundreds of servers in a 512-byte packet or detect a crash quickly, so we need a local LB with health checks." |
| Staff+ | Designs the lifecycle: "active checks for membership plus passive outlier ejection, connection draining on removal so deploys don't kill in-flight requests, and slow start so a cold server isn't flooded and flapped out by least-connections." |
Next: how the load balancer actually chooses.