Load Balancing Algorithms
Why this matters: the algorithm is the load balancer's actual decision. Pick one that ignores server state and a single slow machine silently degrades a fraction of every user's requests.
Key takeaway
Load balancers use specific algorithms to distribute client requests. The important division is static versus dynamic — whether the choice accounts for the current state of the servers. In practice dynamic algorithms give superior results, and the added complexity is usually justified.
The six algorithms
| Algorithm | How it decides | Reach for it when |
|---|---|---|
| Round robin | Forwards requests to servers sequentially in a repeating loop | Servers are identical and requests are uniform |
| Weighted round robin | Each node is assigned a weight; higher-weighted servers receive a larger proportion of requests | Servers have varying capacities |
| Least connections | Assigns each new request to the server with the fewest active connections | Request processing times vary significantly |
| Least response time | Directs traffic to the server with the lowest active response time | Latency is the priority — performance-sensitive services |
| IP hash | Hashes the client's IP address to map them to a specific server | You need session stickiness |
| URL hash | Distributes requests based on the URL | Different clusters serve distinct workloads |
Other variations exist, such as randomized and weighted least connections.
The two that need elaborating
Least connections matters more than it first appears. When request processing times vary significantly — some requests return in 5 ms, others run for 30 seconds — round robin keeps handing new work to a server already tied up with long-running tasks. Least connections prevents that server from becoming overwhelmed, because its high in-flight count makes it unattractive until it drains.
URL hash is about routing to specialized clusters, not balancing across identical ones. It routes traffic to specific server clusters handling distinct services — separating video serving from text, for instance. The servers behind different URL paths are genuinely different fleets with different hardware profiles.
Static versus dynamic
Algorithms are categorized by whether they consider the machine's state:
| Static | Dynamic | |
|---|---|---|
| Example | Round robin | Least connections |
| Basis for the decision | Fixed configuration | Current state of the servers |
| Overhead | Low — simple and cheap | Higher — complexity plus communication |
| Accounts for health and load | ||
| Behavior with a slow server | Keeps sending it traffic | Routes around it |
Static algorithms distribute tasks based on fixed configurations. They are simple and low-overhead but do not account for real-time server health or load.
Dynamic algorithms monitor the current state of servers. They add complexity and communication overhead, but they improve reliability by routing around overloaded or unhealthy machines. Dynamic algorithms require load balancers to exchange information, which makes them modular but complex — and gives them better forwarding decisions through active health monitoring.
The distributed-state problem
Dynamic algorithms have a subtlety worth raising, because interviewers probe it.
"Least connections" is exact when one load balancer sees all traffic. With several load balancers in front of the same pool, each knows only its own connections — so all of them may independently conclude the same server is least loaded and send it a simultaneous burst. This is a herd effect created by the balancing algorithm itself.
Choosing one
| Workload shape | Algorithm | Why |
|---|---|---|
| Identical servers, uniform short requests | Round robin | Cheapest thing that works |
| Mixed server capacities | Weighted round robin | Weights encode the capacity difference |
| Highly variable request duration | Least connections | In-flight count tracks real occupancy |
| Latency-sensitive, heterogeneous fleet | Least response time | Directly optimizes the metric you care about |
| Sticky sessions with no shared store | IP hash | Deterministic client-to-server mapping |
| Distinct workloads behind one domain | URL hash | Routes each path to its specialized cluster |
| Many LBs, large pool | Power of two choices | Near-optimal without global state or herding |
Key takeaway
Start with round robin, move to a dynamic algorithm the moment request durations vary or servers stop being interchangeable — which, at any real scale, is immediately.
Interview signal by level
| Level | What a strong answer sounds like |
|---|---|
| L4 | Names algorithms: "round robin, or least connections." |
| L5 | Matches to the workload: "request durations vary a lot here, so least connections — round robin would pile work onto a server already running a long job." |
| Staff+ | Handles the distributed case: "with several LBs, strict least-connections herds because each has a stale local view. Power-of-two-choices gets near-optimal balance with no shared state. And I'd rather make servers stateless than use IP hash for stickiness." |
Next: what happens when the load balancer has to remember something.