What Load Balancing Is and Why You Need One
Why this matters: every "we'll scale horizontally" answer depends on something routing traffic to the machines you added. That something is a load balancer, and without it extra servers sit idle.
Key takeaway
A load balancer (LB) distributes incoming client requests across a pool of available servers. Its primary job is to divide traffic fairly so no single server crashes under overload — which is what makes adding servers actually increase capacity.
The problem it solves
Typical data centers receive millions of requests per second. To handle that volume, thousands of servers work together to share the load. The question that immediately follows is the one that matters: how are incoming requests distributed across all those available servers?
The load balancer is the first point of contact inside a data center after the firewall. Low-traffic services may not need one at all — but as traffic grows it becomes indispensable.
Why you need one
Three reasons, and they map directly onto the non-functional requirements from the Foundations module:
| Benefit | What it gives you | The NFR it serves |
|---|---|---|
| Scalability | Add servers to increase capacity seamlessly; upscaling and downscaling are transparent to end users | Scalability |
| Availability | If a server fails, the system stays online — the LB detects faults and reroutes traffic to healthy servers | Availability, fault tolerance |
| Performance | Requests go to servers with the lowest load, improving response times and resource utilization | Performance |
Load balancers distribute requests using configured algorithms — round robin, weighted round robin, least response time, least connections, and others covered in Lesson 4. Throughout, they ensure reliability by continuously monitoring server health and directing traffic only to servers that can respond efficiently.
Where load balancers go
Load balancers are commonly deployed between clients and backend servers, but they can also operate between internal service layers. In a typical three-tier architecture they appear at three points:
- Between end users and web servers (or the application gateway).
- Between web servers and application servers.
- Between application servers and database servers.
In reality, a load balancer can sit between any two services that have multiple instances. That is the general rule worth remembering — the three-tier picture is an instance of it, not the definition.
Services offered beyond distribution
A load balancer does considerably more than pick a server:
| Service | What it does | Why it belongs here |
|---|---|---|
| Health checking | Uses the heartbeat protocol to monitor server health and reliability | The LB already talks to every server — it's the natural place to notice one is sick |
| TLS termination | Handles TLS, reducing the processing burden on backend servers | Crypto is expensive; doing it once at the edge frees the whole fleet |
| Predictive analytics | Analyzes traffic patterns to predict usage trends | It sees every request, so it has the best view of demand |
| Reduced human intervention | Automated failure handling cuts manual system administration | Directly lowers MTTR — the availability lever from Foundations |
| Service discovery | Routes requests to the right hosting servers by querying the service registry | Servers come and go; the LB is where that churn is absorbed |
| Security | Mitigates attacks such as denial-of-service at OSI layers 3, 4, and 7 | It's the first thing traffic reaches — the natural choke point |
Overall, load balancers provide flexibility, reliability, redundancy, and efficiency to a design.
Key takeaway
A load balancer is what converts a set of independent servers into a single scalable, self-healing service behind one address. Everything else in this chapter is a decision about how it picks a server and where in the stack it sits.
Interview signal by level
| Level | What a strong answer sounds like |
|---|---|
| L4 | "We put a load balancer in front of the web servers to spread traffic." |
| L5 | Names the services: "it also does health checks and TLS termination, so backends only handle business logic — and it's how we add capacity without clients knowing." |
| Staff+ | Applies it everywhere and owns the SPOF: "LBs between every tier that has multiple instances, not just at the edge. And the LB itself needs redundancy — a pair with a floating VIP, fronted by ECMP at scale." |
Next: balancing across the planet before balancing across a rack.