Polyglot Storage

In one line: the most instructive thing here is not the list of databases. It is that Twitter tried Cassandra, gave up, and built their own — a reversal the design records plainly and most designs would hide.

Twitter employs a polyglot persistence architecture, selecting specific storage models to optimize performance for different services.

Manhattan

To handle rapid user growth, Twitter initially attempted to replace MySQL with Cassandra. However, due to specific limitations, they deprecated Cassandra in 2014 and launched Manhattan, a proprietary, real-time, distributed key-value store.

Manhattan serves as the backend for tweets, accounts, and direct messages. It runs clusters of varying sizes, handling millions of QPS for heavy read/write traffic. Manhattan uses RocksDB as its storage engine for node-level data retrieval.

MySQL to Cassandra to a custom store — read the whole arc, not the endpoint

The sequence matters more than the destination:

MySQL  ->  sharded MySQL (Gizzard)  ->  Cassandra  ->  Manhattan (2014)

Each step was a response to a failure of the previous one, and the Cassandra reversal is the interesting part. Cassandra is a mature, well-regarded, horizontally-scalable store — the obvious answer. Twitter adopted it and then abandoned it.

The design is vague about why ("due to specific limitations"), but the shape of the decision is instructive regardless. Building a database is an enormous undertaking, so an organization only does it when the general-purpose option fails on a dimension they cannot compromise. Publicly discussed reasons for this class of decision typically include operability at multi-tenant scale, predictable latency at high percentiles, and control over storage-engine behaviour.

Two things worth taking from it:

The obvious choice failing is information. If a well-run company adopts the standard tool and then replaces it, the mismatch was structural rather than a matter of tuning.

"Build your own database" is almost never the right answer for you. Twitter had the scale and the engineering budget to amortize it. The lesson is not to imitate the outcome; it is to understand what pressure produces it.

Note also that Manhattan is built on RocksDB rather than from scratch. The reversal was about distribution, coordination, and operability, not about how bytes are written to disk — they kept a proven storage engine and replaced the layer above it. That is the usual shape of a justified build-versus-buy decision: replace the part that does not fit, not the whole stack.

The rest of the stack

Money gets a relational database, and that is the pattern

Note precisely which workloads use MySQL and PostgreSQL: "services requiring strong consistency, such as ad exchanges and campaign management."

Not tweets. Not timelines. Not the follow graph. Money.

That is the same conclusion that building block reached — in-progress trips and payments in MySQL, everything high-volume in Cassandra — and the same one that building block's evaluation reached about billing.

The reason is consistent across all three: financial data is low-volume, high-consequence, and transactional. An ad campaign has a budget that must not be overspent; an auction must not double-charge. Those need atomicity and isolation, and they involve thousands of rows rather than billions.

Volume and consistency requirements are usually inversely correlated, which is what makes polyglot persistence work at all. If the data that needed transactions were also the data with the highest volume, you would have a genuinely hard problem. Usually it is not.

FlockDB exists because the follow graph is a different shape entirely

Twitter models relationships (followers, following, block lists) as a graph, stored in FlockDB, optimized for large adjacency lists and rapid traversal.

Why not just a table of (follower_id, followee_id)? Because of the queries Lesson 5 needs:

"Who follows account X?"   -> adjacency list, potentially 50 MILLION entries
"Who does user Y follow?"  -> adjacency list, typically hundreds
"Does A follow B?"         -> point check, needed on every protected-tweet read

The first is the killer. Fan-out on write means enumerating a 50-million-entry adjacency list, in pages, quickly. A relational index can do it, but a store designed around adjacency lists does it better — and the follow graph is queried on essentially every write and many reads.

Two properties make this graph unusual even among graphs:

It is extremely skewed. Median follower count is in the hundreds; the maximum is in the tens of millions. Any storage design assuming uniform node degree fails at the top — the same power-law problem as Lesson 5's celebrities and Lesson 10's heavy hitters.

It is directed and asymmetric. Unlike a friendship graph, following is one-way, so followers(X) and following(X) are different lists with wildly different sizes. Both must be indexed independently.

When your access pattern is "enumerate this node's neighbours, fast, at any scale," you want a graph store rather than a join. That is the whole justification, and it is enough.

Kafka to Google Cloud — 400 billion events a day

Twitter processes approximately 400 billion real-time events daily. Events are processed on-premise using Kafka, then moved to Google Cloud via an event processor that converts Kafka topics to Cloud pub-sub topics. Google Dataflow jobs handle deduplication and real-time aggregation. Results are stored in BigQuery for analysis and Bigtable for serving.

Lesson 3 converted the headline figure: 4.63 million events per second — 267 times the tweet-posting rate. Every like, view, impression, and scroll is an event.

Two things stand out.

Deduplication is called out explicitly, which tells you the pipeline is at-least-once. Kafka delivers duplicates under retry, so dedup is not an optimization but a correctness requirement — the same conclusion that building block reached about payment intents. In any at-least-once pipeline, deduplication is part of the contract.

The results land in two stores, and the split is by access pattern. BigQuery for analysis — ad-hoc, columnar, scan-heavy. Bigtable for serving — point lookups on the request path. Same data, two shapes, because analytical and operational queries are different workloads. That is the same instinct as the whole polyglot argument, applied one level down.

The partly-cloudy migration is a good example of splitting by criticality

In 2018, Twitter adopted a partly cloudy strategy, migrating data from on-premise Hadoop clusters to Google Cloud. Initially, they moved ad-hoc clusters and cold storage, while keeping real-time production clusters on-premise.

Notice which half moved. Ad-hoc analytics and cold storage — workloads that are latency-tolerant, bursty, and cheaper to rent than to own. Real-time production stayed put — predictable load, latency-critical, cheaper to own at steady state.

That is the correct decomposition, and the reasoning generalizes: rent the bursty and latency-tolerant, own the steady and latency-critical. Cloud elasticity is worth paying for when demand varies; at constant high utilization it is not.

It is also a reversible migration path. Moving the cold half first means the risky half stays under your control while you learn — the same instinct as any incremental rollout.

The design's own question: why keep relational databases at all?

The design asks: "Why does Twitter still keep relational databases like MySQL or PostgreSQL when it has several other specialized storage systems?"

Three reasons, in order of weight:

Transactions. Nothing else in the stack offers them. An ad auction that charges an advertiser and decrements a budget must do both or neither. Manhattan is a key-value store; FlockDB is a graph; neither will atomically update two related rows.

Joins and ad-hoc queries. Campaign management is a genuinely relational workload — advertisers, campaigns, creatives, targeting rules, spend — with queries nobody anticipated at schema-design time. That is precisely what SQL is for.

The volume does not require anything else. This is the underrated reason. Ad campaigns number in the millions, not billions. A workload that fits comfortably in a relational database should stay in one, because you get transactions, joins, and mature tooling for free.

The general principle: specialized stores earn their place by necessity, not by preference. Every one of them costs operational overhead, a new failure mode, and engineers who understand it. You adopt one when the relational answer genuinely fails — which for ad campaigns, it does not.

The justification worth giving: these are not four databases because four are fashionable, but because immutable append-only tweets, relationship traversal, ephemeral hot timelines, and inverted-index retrieval have genuinely incompatible access patterns. Forcing them into one store means the strictest requirement wins everywhere.

Next: search over a trillion records.