Interview Walkthrough: Design a Globally Replicated User Profile Service

This is a complete worked interview. The prompt is deliberately mundane — "Design a service that stores and serves user profiles for a global product" — because the difficulty is not in the feature list. It is entirely in replication, consistency, and failure, which is exactly what this chapter taught.

Step 0 — Scope it before you design

Do not draw a box. Ask.

• Scale? How many users, what read and write rates, what read/write ratio?
• Geography? One region or several? Where are the users?
• What is in a profile? Display name and avatar, or email, phone, and privacy settings? The answer changes the consistency requirement completely.
• Who reads it? Only the owner, or every other service in the company on every request?
• How fresh must a read be? Is a 5-second-stale display name acceptable? Is a 5-second-stale privacy setting acceptable?
• Compliance? Data residency requirements — must EU user data stay in the EU?

Then state the assumptions you will design against, so the interviewer can correct you cheaply:

"I'll assume 500 million users, 4 regions, a read/write ratio around 1000:1 — roughly 500k profile reads per second globally and about 500 writes per second. Profiles include display name, avatar, email, and privacy settings. Most reads come from other internal services on the request path, so profile reads sit in the critical path of nearly every page. I'll target p99 under 50 ms for reads."

1. Functional Requirements

Deliberately small:

• Create a profile.
• Read a profile by user ID.
• Update fields on a profile.
• Bulk-read profiles by ID (the feed needs 50 authors at once).

Say the quiet part: "The functional surface here is four operations. Everything hard about this problem is non-functional — where the data lives, how fresh a read is, and what happens when a region is cut off. Let me spend my time there."

2. Non-Functional Requirements — the heart of this problem

Do not list adjectives. Turn each requirement into a design force.

The consistency row is where this interview is won. Split the data:

"These are not the same data. I'm going to design one system with two consistency paths rather than paying linearizable prices for avatar URLs, or accepting eventual consistency on a block list."

3. Core Entities & Key State

For an infrastructure problem, ask what state exists and where it lives rather than drawing an ER diagram.

The key entity:

Profile
  user_id        (partition key)
  home_region    -- residency + write routing
  display_name, avatar_url, bio     -- eventual
  email, phone                      -- eventual
  privacy_settings, account_status  -- strong
  version        -- monotonically increasing; the read-your-writes watermark
  updated_at

That version field is not decoration. It is the watermark from Lesson 7 — returned on every write, echoed on every read that needs freshness. Point at it explicitly; interviewers notice.

4. API & Interfaces

Two distinct surfaces, and confusing them is the classic junior mistake.

Internal (service to service) — gRPC:

GetProfile(user_id, min_version?) -> (Profile, version)
BatchGetProfiles([user_id], consistency: EVENTUAL | STRONG) -> [Profile]
UpdateProfile(user_id, fields, idempotency_key) -> (version)

External (client-facing) — REST behind the gateway:

GET   /v1/users/{id}
PATCH /v1/users/me          Idempotency-Key: <uuid>

Three deliberate choices to narrate:

• min_version on the read. The caller says how fresh it needs to be. A replica that has not reached that version waits briefly or forwards to the leader. This is read-your-writes as an API parameter rather than a routing hack.
• consistency on the batch read. Rendering a feed? Eventual. Enforcing a block list? Strong. The caller knows which; the storage layer cannot guess.
• idempotency_key on the write. Lesson 4. Without it, a retried UpdateProfile after a lost response can clobber a concurrent edit.

5. High-Level Design

Start dead simple, then break it deliberately.

(a) The naive core

Correct, and it fails every NFR: a user in Singapore pays 200 ms to reach a US primary, and one database outage takes down every product surface that renders a name.

(b) Add caching and read replicas

Given 1000:1 reads, the first move is obvious.

This buys throughput and immediately introduces the bug from Lesson 7: a user updates their name, the read goes to a lagging replica or a stale cache, and the change appears to vanish.

"The fix is read-your-writes, not strong consistency. On write I return the new version and set the cache entry synchronously; the client passes min_version back and any replica below it forwards to the primary. I get correct-feeling behavior without a quorum round trip on 500k reads per second."

(c) Go multi-region

Every region serves reads locally, so p99 stays under 50 ms. Writes route to the user's home region — which satisfies data residency and, critically, gives every profile a single writer, sidestepping concurrent multi-region write conflicts entirely.

(d) The two-path write, and what a partition does

Now answer CAP honestly, per path:

"Under a region partition the two paths behave differently on purpose. Profile reads stay available everywhere serving possibly-stale data — AP, because a slightly old avatar is harmless and a blank profile page breaks every product surface. Safety writes — blocks and bans — go CP: if the partitioned side can't reach a quorum it rejects the write and returns an error, because a block that silently didn't apply is a user-safety incident. Cosmetic writes in a partitioned region I'd queue locally and replicate on heal."

That paragraph is the whole interview. It names the failure, splits the decision by data, and states who pays.

6. Deep Dives & Follow-up Questions

"A user updates their name and doesn't see it. Walk me through the fix."

Read-your-writes. The write returns a monotonic version; the client echoes it as min_version on subsequent reads. A replica at a lower version either waits briefly for replication or forwards to the primary. I'd key the watermark to the user, not the connection, so it survives a device switch — the cookie-based sticky-session version breaks the moment they open the app on their phone. I would not solve this with strong consistency; it's a per-session guarantee and paying quorum latency on every read to fix it would be a large regression for a small class of requests.

"Cache invalidation across three regions — how?"

A change stream out of the home region's database fans out to each region's invalidator, which deletes the key locally. It's asynchronous, so there's a window where a region serves a stale profile — bounded by replication lag, typically well under a second, and acceptable for cosmetic fields. For safety fields I don't rely on invalidation at all: I write through the cache synchronously as part of the quorum write, because "we'll invalidate it shortly" is not an acceptable story for an unblock that didn't take effect. I'd also put a short TTL under everything as a backstop, so a dropped invalidation message self-heals instead of serving stale data forever.

"A region goes down. What happens?"

Reads: other regions keep serving from their replicas, so the read path degrades to slightly staler data rather than failing. Writes for users homed in the dead region are the real problem — they have a single writer, and it's gone. I'd fail their home region over to a designated secondary, which requires the control plane to update the home-region mapping. That mapping is consensus-backed precisely so two regions can never simultaneously believe they own the same user's writes. Failover must use a fencing token: if the original region is only partitioned and comes back believing it's still the writer, storage rejects its stale-epoch writes. Without fencing, this is a textbook split brain.

"How do you know the region is actually down and not just slow?"

I don't, and that's the point — the models are indistinguishable from outside. I'd use a phi-accrual style detector that outputs a suspicion level rather than a boolean, so the failover threshold is tunable instead of a single brittle timeout. And I'd deliberately make failover slow and manual-ish for writes: an unnecessary failover of a healthy-but-slow region costs more than a few extra minutes of write unavailability. Meanwhile reads keep working throughout, which is what buys me the time to be careful.

"Your batch read fetches 50 profiles. What's the p99?"

If I fan out to 50 backends and wait for all of them, my p99 is dominated by tail latency amplification — with a per-call p99 of 20 ms, the odds all 50 stay fast are about 60%, so the batch p99 lands far above 20 ms. Fixes, in order: serve batches from a single multi-get against the cache so it's one call rather than 50; hedge the slow tail by re-issuing to a second replica at the p95 mark (safe — reads are idempotent); and return partial results with a deadline rather than blocking the whole page on the slowest profile.

"What if the profile service returns corrupted data?"

That's a Byzantine failure, and inside our own datacenter I don't design full BFT for it — 3f+1 replicas and signed messages are for adversarial participants. I do defend against the realistic subset: checksums on stored records to catch bit rot and silent disk corruption, schema validation at the service boundary so a bad deploy fails loudly instead of writing garbage, and canary deploys so a corrupting version reaches 1% of traffic rather than 100%.

"Traffic triples during a launch. What breaks first?"

Cache hit rate, because a launch brings profiles that aren't warm. Misses hit the replicas, replica latency climbs, and that looks like a temporal failure rather than an outage — everything green, everything slow. Defenses: propagated deadlines so we shed rather than queue, a concurrency limit at the service so we fail fast instead of collapsing, and a circuit breaker on the replica path so a struggling replica gets room to recover. I'd also alert on client-observed success rate rather than server health checks, since this is exactly the gray-failure shape where the dashboards say healthy and the users disagree.

Now do it live

The next section drills the same concepts as rapid-fire probes — the exact questions an interviewer uses to find the edge of your understanding.