Stateful vs. Stateless Load Balancing
In one line: the moment a client must keep reaching the same server, the load balancer stops being a simple dispatcher and starts being a distributed system with its own consistency problem.
The precondition worth noticing
Read that framing carefully, because it contains the most important design guidance in this lesson: the LB only has to manage session state if something below it does not.
If you push session state into a shared store, every server can serve every request, and the load balancer is free to use whatever algorithm balances best. Everything below is what you deal with when you cannot do that.
Stateful load balancing
Stateful load balancing maintains a record of the sessions between clients and servers, and uses that state to guide routing decisions. In essence, the LB keeps a mapping of incoming clients to backend servers.
The cost is significant: this increases complexity and limits scalability, because session information must be synchronized across all load balancers to ensure consistent forwarding. Add a second LB and it needs the same table; add a tenth and you are running a replicated database in your traffic path, on the hot path, with all the consistency problems that implies.
Stateless load balancing
Stateless load balancing does not store session state. It typically uses consistent hashing to route requests: the client identifier is hashed, and the hash determines the server.
It is faster and lighter — there is no table to store, replicate, or keep consistent, because the routing decision is recomputed from the key every time. Two load balancers that never talk to each other will independently reach the same answer for the same client.
The weakness: it handles infrastructure changes such as scaling less gracefully than a stateful design. To mitigate that, a local state is often combined with consistent hashing.
Why consistent hashing, and not plain modulo
The naive approach — server = hash(client) % N — works until N changes. Add or remove one server and almost every client remaps to a different one, because the modulus changed for everybody. Every session breaks at once.
Consistent hashing places servers and keys on a ring, so adding or removing a server only remaps the keys in that server's arc — roughly 1/N of them. The rest stay exactly where they were. That property is what makes stateless load balancing survivable during scaling, and it is the same mechanism key-value stores uses for partitioning.
The definition that actually distinguishes them
The dividing line is more precise than "does it store anything," and it is worth quoting exactly:
If the state is synchronized across multiple load balancers, it is considered stateful. If state is maintained locally within a single load balancer, or derived algorithmically, it is considered stateless.
This is subtler than most people expect. A load balancer holding a local cache of recent routing decisions is still stateless by this definition, because nothing has to be replicated for correctness. What makes a design stateful is the synchronization requirement — the need for every LB to agree.
That is the real cost being described. Not memory. Coordination.
| Aspect | Stateful | Stateless |
|---|---|---|
| How routing is decided | Lookup in a session table | Computed from a hash of the key |
| Shared across LBs | Must be synchronized | Nothing to share |
| Speed and weight | Heavier — lookup plus replication | Faster and lighter |
| Scaling the LB tier | Hard — more LBs, more sync | Trivial — LBs are independent |
| Scaling the server pool | Handled gracefully — the map is explicit | Less graceful — remapping shifts some keys |
| Failure of one LB | Its sessions must exist elsewhere | Any LB computes the same answer |
Note the two scaling rows point in opposite directions, and that is the genuine trade: stateful handles server pool changes gracefully but makes the LB tier hard to scale; stateless is the reverse.
The answer interviewers are hoping for
Asked "stateful or stateless," the strongest reply questions the premise:
"First I'd ask why the LB needs session state at all. If I can push sessions into a shared cache or a signed token the client carries, the servers become stateless, any server can serve any request, and the LB doesn't need affinity — which removes the entire problem. If I'm stuck with server-local sessions, I'd go stateless with consistent hashing, because synchronizing a session table across the LB tier puts a replicated database in the hot path."
That answer demonstrates you know the trade-off and that the best move is usually to avoid needing it.
Key takeaway
The cost of stateful load balancing is coordination, not storage. Stateless designs replace a synchronized table with a computation — and consistent hashing is what makes that computation stable when the fleet changes.
Interview signal by level
| Level | What a strong answer sounds like |
|---|---|
| L4 | "Sticky sessions keep a user on the same server." |
| L5 | Knows the mechanism: "stateless with consistent hashing, so we don't have to replicate a session table between load balancers." |
| Staff+ | Removes the requirement: "I'd push session state into a shared cache so the servers are stateless and affinity isn't needed at all. Failing that, consistent hashing — because the real cost of stateful isn't memory, it's synchronizing state across every LB on the hot path." |
Next: how deep into the request the load balancer is allowed to look.