Stateful vs. Stateless Load Balancing
Why this matters: the moment a client must keep reaching the same server, the load balancer stops being a simple dispatcher and starts being a distributed system with its own consistency problem.
Key takeaway
Load balancers may need to handle session persistence between clients and hosting servers. If session information is not stored in a lower layer — a distributed cache or database — the LB itself must manage it, and there are two ways to do that: stateful and stateless.
The precondition worth noticing
Read that framing carefully, because it contains the most important design guidance in this lesson: the LB only has to manage session state if something below it does not.
If you push session state into a shared store, every server can serve every request, and the load balancer is free to use whatever algorithm balances best. Everything below is what you deal with when you cannot do that.
Stateful load balancing
Stateful load balancing maintains a record of the sessions between clients and servers, and uses that state to guide routing decisions. In essence, the LB keeps a mapping of incoming clients to backend servers.
The cost is significant: this increases complexity and limits scalability, because session information must be synchronized across all load balancers to ensure consistent forwarding. Add a second LB and it needs the same table; add a tenth and you are running a replicated database in your traffic path, on the hot path, with all the consistency problems that implies.
Stateless load balancing
Stateless load balancing does not store session state. It typically uses consistent hashing to route requests: the client identifier is hashed, and the hash determines the server.
It is faster and lighter — there is no table to store, replicate, or keep consistent, because the routing decision is recomputed from the key every time. Two load balancers that never talk to each other will independently reach the same answer for the same client.
The weakness: it handles infrastructure changes such as scaling less gracefully than a stateful design. To mitigate that, a local state is often combined with consistent hashing.
The definition that actually distinguishes them
The dividing line is more precise than "does it store anything," and it is worth quoting exactly:
If the state is synchronized across multiple load balancers, it is considered stateful. If state is maintained locally within a single load balancer, or derived algorithmically, it is considered stateless.
This is subtler than most people expect. A load balancer holding a local cache of recent routing decisions is still stateless by this definition, because nothing has to be replicated for correctness. What makes a design stateful is the synchronization requirement — the need for every LB to agree.
That is the real cost being described. Not memory. Coordination.
| Aspect | Stateful | Stateless |
|---|---|---|
| How routing is decided | Lookup in a session table | Computed from a hash of the key |
| Shared across LBs | Must be synchronized | Nothing to share |
| Speed and weight | Heavier — lookup plus replication | Faster and lighter |
| Scaling the LB tier | Hard — more LBs, more sync | Trivial — LBs are independent |
| Scaling the server pool | Handled gracefully — the map is explicit | Less graceful — remapping shifts some keys |
| Failure of one LB | Its sessions must exist elsewhere | Any LB computes the same answer |
Note the two scaling rows point in opposite directions, and that is the genuine trade: stateful handles server pool changes gracefully but makes the LB tier hard to scale; stateless is the reverse.
Key takeaway
The cost of stateful load balancing is coordination, not storage. Stateless designs replace a synchronized table with a computation — and consistent hashing is what makes that computation stable when the fleet changes.
Interview signal by level
| Level | What a strong answer sounds like |
|---|---|
| L4 | "Sticky sessions keep a user on the same server." |
| L5 | Knows the mechanism: "stateless with consistent hashing, so we don't have to replicate a session table between load balancers." |
| Staff+ | Removes the requirement: "I'd push session state into a shared cache so the servers are stateless and affinity isn't needed at all. Failing that, consistent hashing — because the real cost of stateful isn't memory, it's synchronizing state across every LB on the hot path." |
Next: how deep into the request the load balancer is allowed to look.