Cheat Sheet

Key terms

The formulas

Availability = Uptime / (Uptime + Downtime)
Availability = MTBF / (MTBF + MTTR)

MTBF = (Total Elapsed Time - Sum of Downtime) / Number of Failures
MTTR = Total Maintenance Time / Number of Repairs

Serial dependencies:   A_total = A_1 * A_2 * ... * A_n
Redundant components:  A_total = 1 - (1 - A_single)^n

Error budget = 1 - SLO
Concurrency  = Arrival rate * Latency        (Little's Law)

The nines

Serial dependencies multiply: 4 services at 99.9% → 99.6% (~35 h/year). Redundancy multiplies unavailability: 2 nodes at 99% → 99.99% — only if failures are independent.

Reliability vs availability

Severed-network data center: availability 0%, reliability 100%. The dangerous quadrant is high A, low R — always reachable, often wrong.

Fault tolerance

• Redundancy sizing: N+1 (any single failure) · 2N (whole set/DC) · 2N+1 (both)
• Utilization ceiling to survive f of n: (n - f) / n — 3 nodes, 1 failure → 66% max
• Replication: sync = strong + less available · async = available + eventual · semi-sync = bounded loss
• Redundancy does not protect against software bugs — that's canaries, staged rollout, rollback
• Contain blast radius: cells, bulkheads, graceful degradation, static stability

Checkpointing

A cut is consistent iff there are no orphan messages.

Uncoordinated checkpoints → domino effect (cascading rollbacks). Fix: coordinated snapshots (Chandy-Lamport) or message logging.

Scalability

Vertical (up)    = bigger machine  — simple, hard ceiling, downtime, costly at the top
Horizontal (out) = more machines   — no ceiling, fault tolerant, coordination complexity
Autoscaling      = demand-driven   — good for cycles, BAD for flash spikes

Three dimensions: size · administrative · geographical.

What limits scaling: contention (queueing on a shared resource — flattens the curve) and coherency (sync cost between nodes — eventually bends it down). Precondition for scaling out: statelessness.

The scaling ladder

1 server → split app/DB → cache → load balance → replicas + CDN
       → async workers + queues → shard → decompose into services

Cheapest and least invasive first. Sharding and microservices last — the partition key is the decision you can't cheaply reverse.

Performance

• Latency = one request. Throughput = requests/sec. Batching trades one for the other.
• Use percentiles, not averages. p99 is the SLO; p99.9 is often your biggest customers.
• Tail is amplified by fan-out — 100 calls at p99 = 100 ms → ~63% of requests hit a slow path.
• Levers: caching (precompute only what has a reader) · algorithm/data-structure fit (check the write rate before picking a read-optimized index) · load distribution.

Back-of-the-envelope

Average QPS = DAU * actions per user per day / 100,000   (~86,400, rounded)
Peak QPS    = Average QPS * peak factor (2x-10x)
Storage     = writes/day * bytes/write * retention * replication factor
Egress      = QPS * average response size

Then read each number as a decision — 300 GB/sec of egress makes a CDN mandatory, not optional.

Quick decision cues

• Load is growing / requests surging → scalability
• Must be reachable 24/7 → availability
• Must keep running through component failure → fault tolerance
• Must produce correct results consistently → reliability
• Nobody can intervene (spacecraft, embedded) → fault tolerance
• Money, ledgers, inventory → consistency first, latency last
• Target above three nines → recovery must be automated
• Read-heavy → cache + replicas · Write-heavy → shard + queue
• Known spike at a known time → pre-provision; autoscaling is too slow
• Symptom is delay, cause is load → fix the load, not the symptom

Work the Interview Walkthrough for these applied end to end, and the Assessments & Drills for the scenario-matching practice.