The Message Service and Transient Storage

In one line: "once delivered, the message is deleted" is one sentence, and it is the most distinctive architectural claim in the chapter.

The message service

The WebSocket server communicates with the Message service, which manages message storage using a Mnesia database cluster. The service acts as an interface for other services to store and retrieve messages. It deletes messages after a configurable period and provides APIs to filter messages by user ID or message ID.

Delivery

1. User A sends the message to their connected WebSocket server.
2. A's server queries the WebSocket manager to locate B's server. If B is online, the manager returns the connection details.
3. Simultaneously, the message is sent to the message service and stored in Mnesia for FIFO processing. Once delivered, the message is deleted from the database.
4. A's server forwards the message to B's server.
5. If B is offline, the message remains in Mnesia. When B comes online, pending messages are delivered via push notifications. Undelivered messages are permanently deleted after 30 days.

'Once delivered, the message is deleted' — a privacy posture expressed as a retention policy

This inverts what every other chapter in this module does. Tweets, photos, videos, posts, and crawled pages are all stored permanently and served repeatedly. Here the server's copy exists only between send and delivery, and then it is gone.

Three consequences, and the third is the point.

Storage is a queue depth, not an archive. Lesson 3's 300 TB is the 30-day tail of undelivered messages. The vast majority occupy the server for seconds, which is why a service with two billion users needs less storage than one with a hundred million photos.

The server is a relay, not a repository. It has no chat history, so it cannot search it, analyse it, or serve it to a new device.

A server that does not retain cannot be compelled to produce. That is the actual motivation, and it is a legal and political design decision expressed as a data-retention policy. Combined with Lesson 2's end-to-end encryption, the claim becomes: even the ciphertext is gone once it is delivered.

Deleting data is a feature when the threat model includes your own operator. No other chapter in this module has a requirement of that shape, and it is worth recognizing as a category — the design is protecting users from the service, not just from outsiders.

And it contradicts the multi-device requirement, directly

Lesson 2 flagged this and here is where it bites.

Requirement: "ensure chat history is consistent across all user devices."

This lesson: "Once delivered, the message is deleted from the database."

Delivered to whom? If A sends a message and B's phone receives it, the message is deleted — and B's laptop, which was offline, will never see it. The history the requirement demands does not exist anywhere the second device can reach.

The available resolutions, none of which the chapter picks:

Deliver to every registered device before deleting. Now "delivered" means all devices, and a tablet left off for a month holds messages on the server for a month. The queue depth is set by the least-available device, which breaks Lesson 3's storage assumption.

Device-to-device sync. The primary device transfers history directly to a new one. Works, requires both online simultaneously, and is what real products actually do.

Encrypted server-side backup. The server stores ciphertext it cannot read. Preserves the privacy posture and reintroduces the storage the design was avoiding.

When a requirement demands state and the design deletes it, one of them must change. Naming which — and I would drop the multi-device consistency requirement or move it to device-to-device sync — is a stronger answer than noticing the conflict.

Mnesia is a hint about the whole stack

The choice of Mnesia is not incidental. It is Erlang's built-in distributed database, and its appearance tells you the messaging tier is an Erlang system.

That connects directly to Lesson 3's 10 million connections per server. Erlang's runtime is built around enormous numbers of lightweight processes with isolated heaps and preemptive scheduling — which is precisely the model that makes millions of mostly-idle connections tractable, where a thread-per-connection design dies in the thousands.

So the stack is coherent rather than assembled:

Erlang processes  -> millions of cheap concurrent connections
Mnesia            -> distributed store in the same runtime, no serialization boundary

Mnesia's specific fit here is worth naming: it holds data in memory with optional disk persistence, replicates across nodes, and lives inside the application process — so storing a message is not a network hop to a separate database tier.

For a store-and-forward buffer where most rows live for seconds, an in-memory replicated store is exactly right. You are not building an archive; you are building a queue that survives a node failure.

When a database choice looks unfamiliar, check whether it belongs to the same runtime as the application — the answer often explains the whole architecture.

'Simultaneously' in step 3 undoes Lesson 4's ordering

Step 3 says the message is "simultaneously sent to the message service and stored" while step 4 forwards it to B.

Lesson 4 established the correct order: store, then deliver — because a crash between them then duplicates rather than loses.

"Simultaneously" is weaker. If the store and the forward are genuinely concurrent, a crash after the forward and before the store commits means the message reached B and was never durable — which is survivable — but a crash after neither means A got its sent acknowledgement for a message that no longer exists anywhere.

The safe reading is that the sent acknowledgement in Lesson 4's step 3 is issued only once Mnesia has the message, and the forward proceeds in parallel afterwards. That preserves the guarantee while keeping the delivery path fast.

"Simultaneously" is not an ordering, and in a store-and-forward system the ordering is the guarantee. Worth pinning down rather than accepting.

Thirty days, and what happens after

Undelivered messages are permanently deleted after 30 days.

A bounded queue depth, which is what makes Lesson 3's storage estimate finite. Without it, a user who never returns accumulates messages forever.

Two things worth naming.

The sender is never told. A message that expires after 30 days simply stops existing, and A's client still shows a single tick. Silently dropping data after a month is a product decision that should be visible, and is not.

It bounds the multi-device problem too. Whatever resolution you pick for the previous callout, the storage cost is capped at 30 days — so "deliver to all devices before deleting" is expensive but not unbounded.

Ordering is a place to be honest about what you are actually guaranteeing. Client clocks cannot be trusted and messages reach different servers at different times, so the achievable property is a single consistent order across devices, not true causal order. Server-side timestamps from NTP-synced hosts give exactly that, and the occasional message that appears slightly out of send order is a trade users do not notice — while two devices disagreeing about order is immediately visible.

Next: group messages, and where end-to-end encryption breaks.