Technology Choices and Long Polling

In one line: the language split — Python for most things, C++ for the hot path — is a pattern you will meet repeatedly, and long polling is the classic answer to "how do we push updates without WebSockets?"

Two languages

Optimize the few percent that matters, and keep the rest productive

This is a specific application of a general rule: most code is not performance-critical, and the code that is, matters enormously.

Feature extraction is the right thing to write in C++ — it runs per request on the recommendation path, it is compute-bound, and Lesson 6 established that latency there is already a problem. A constant-factor speedup on that component is directly visible to users.

Writing the whole application in C++ to get it would be a poor trade: most endpoints are I/O-bound, waiting on MySQL or Memcached, where language speed is irrelevant. You would pay enormous development cost for no user-visible gain.

The complementary point is the cost of the split: two toolchains, two skill sets, a serialization boundary, and Thrift to maintain. That is only worth paying where the speedup is real.

The Blob Store chapter mentioned the same instinct — "different languages handle specific tasks (e.g. C for encryption)" — and it is the same reasoning. Profile, then optimize the hot path in a faster language; leave the rest alone.

Thrift does more than serialize — it isolates failures

Lesson 11's evaluation adds something the technology section does not: "Thrift isolates services and therefore failures."

That reframes it from a serialization format to a service boundary. Because the Python application and the C++ feature extractor communicate over a defined RPC interface rather than being linked into one process:

• A crash in feature extraction does not take down the web tier — the call fails and can be handled.
• Each side can be deployed and scaled independently.
• The interface is explicit and versioned, so changes are visible rather than implicit.

So the cross-language boundary, which looked like a cost, also buys fault isolation. That is worth noticing: a boundary imposed for one reason often provides a second benefit, and here the language split forced a service boundary that improves resilience.

Polling versus long polling

Real-time features (comments, votes) require frequent updates. Standard polling overburdens the server with empty checks. Quora uses long polling: the server holds the request open (up to 60 seconds) until an update is available, ensuring immediate response without wasted cycles.

Polling's problem is that almost every request is wasted

With short polling, the client asks "anything new?" on a fixed interval. Most of the time the answer is no.

That has two costs, and the dial that controls them moves them in opposite directions:

• Short interval → low latency, but most requests return empty — wasted round trips, wasted server capacity, wasted battery on mobile.
• Long interval → fewer wasted requests, but updates are stale by up to one interval.

Long polling removes the trade. The client asks once and the server holds the connection open — up to 60 seconds here — responding the instant content appears. You get the latency of a very short poll with the request volume of a very long one.

This is exactly message-queue design's argument for long polling over short polling in message consumption. Same mechanism, same reasoning: hold the request rather than repeating it.

Long polling is not free — each open request holds server resources

"What is the main advantage of using long polling instead of polling?"

In long polling, the client sends a request that the server holds open until new data becomes available. This reduces the number of repeated polling requests. However, long polling is resource-intensive because each open request consumes server resources such as sockets, threads, or memory until it completes.

WebSockets are another low-latency solution with low overhead. However, WebSockets might be overkill for the features offered by Quora.

The cost is real and worth stating: with a million concurrent users, you hold a million open connections, each consuming a socket and some memory. That is a very different server profile from handling short, completed requests.

It is also the C10K problem the YouTube chapter met when choosing an event-driven web server — many concurrent, mostly-idle connections is precisely the workload that breaks thread-per-connection models.

And note the honest assessment of the alternative: WebSockets would be lower overhead and give true bidirectional push, but they are "overkill" here. Quora's real-time needs are modest — new comments and vote counts, not a chat application or a collaborative editor.

Choose the lightest mechanism that meets the requirement. Long polling sits between short polling and WebSockets in both capability and complexity, and Quora's requirement sits there too.

Memcached multiget

Memcached uses multiget() to retrieve multiple keys simultaneously, reducing latency.

Batching turns N round trips into one — the same insight as Redis pipelining

A page needs many cached values: the question, several answers, their vote counts, author details. Fetching them one at a time is N sequential round trips, and the page waits for all of them.

multiget() fetches them in one request. Total latency becomes one round trip plus the server-side lookups, rather than N round trips.

This is distributed caching's pipelining result — "a 5x throughput gain even on loopback, which tells you the bottleneck is per-request syscall overhead rather than the network." When per-item work is tiny, the ceremony around each request dominates, and batching amortizes it.

Lesson 11's evaluation lists it among performance techniques for exactly that reason. It also composes well with Lesson 4's partial page generation: a manager can identify everything a page needs and fetch it in one batch rather than letting each worker fetch independently.

Note: Quora uses AWS infrastructure, including S3 and Redshift for data warehousing.

Interview signal by level

Next: checking the design against the requirements.