The Cost of Servers

In one line: an estimate in servers is abstract. The same estimate in money is a decision — and the gap between the plausible and pathological peak scenarios turns out to be the difference between a rounding error and five billion a year.

The reference instance

To estimate cost, take an AWS EC2 m7i.16xlarge — a 64-core, 256 GB machine on a 4th-generation Intel Xeon, matching the reference server from Lesson 2:

Instance size	vCPU	Memory (GiB)	Instance storage	Network bandwidth	EBS bandwidth
m7i.16xlarge	64	256	EBS-only	25 Gbps	20 Gbps

Hourly cost with a one-year contract: 3.54816 USD per hour.

What each scenario costs

Applying that rate to the three server counts from the previous lesson:

Scenario	Servers	Cost per hour	Cost per year
Lower bound (uniform traffic)	2	7.10 USD	~62,000 USD
Pareto 80/20 assumption	8	28.38 USD	~249,000 USD
Peak load (all at once)	157,000	557,061 USD	~4.88 billion USD

2       * 3.548 =       7.096 USD/hour
8       * 3.548 =      28.38  USD/hour
157,000 * 3.548 = 557,061     USD/hour

Costs escalate quickly in the peak load scenario.

Read the annual column again

Provisioning for the pathological "everything arrives at once" bound costs roughly 4.88 billion USD per year — against about 249,000 USD for the Pareto-based plan. That is a factor of ~20,000 in spend to defend against a traffic pattern that does not occur.

This is the clearest possible argument for why peak modeling matters and why the answer to a genuine flash crowd is graceful degradation, not provisioning. No business would approve the first number, so a design that requires it isn't a design.

Cost as a design force

Money behaves exactly like the NFRs from the previous chapter: it constrains architecture, and pretending otherwise produces designs that never ship.

Cost driver	What inflates it	The lever
Compute	Provisioning for peak rather than average	Autoscale, use spot capacity for batch, degrade under extremes
Storage	Retaining everything forever at full replication	Tier cold data to cheaper classes; set retention policies
Egress	Serving bytes from origin	CDN — usually the single largest saving available
Cross-AZ / cross-region traffic	Chatty services spread across zones	Co-locate communicating services; batch calls
Idle redundancy	Hot standby for everything	Match failover mode to the actual availability target

Say this in an interview

"Let me price this. Eight instances at roughly 3.50 USD an hour is about 250,000 USD a year, which is reasonable for a service this size. The peak-load version would be nearly five billion, which tells me provisioning for that scenario is off the table — so I'll plan for the realistic peak and handle true flash crowds by degrading."

Very few candidates cost their designs. It reads as ownership, because in a real organization someone has to sign for it.

A caveat on cloud pricing

These figures use on-demand-style pricing with a one-year commitment. Real bills differ, sometimes substantially:

Reserved capacity and savings plans cut compute meaningfully for predictable baseline load.
Spot instances are far cheaper for interruptible work — batch jobs, transcoding, analytics.
Volume discounts are routine at scale; large organizations do not pay list price.
Egress is frequently the surprise line item, and it is not in the instance price at all.

For a BOTEC, list price is the right choice: it is public, defensible, and conservative. Note the caveat, then use it.

Key takeaway

Convert servers to money before you defend a design. The cost delta between assumptions is often larger than the technical difference between architectures — and it is the number a business will actually make a decision on.

Interview signal by level

Level	What a strong answer sounds like
L4	Never mentions cost.
L5	Acknowledges it: "we should use reserved instances since the baseline load is predictable."
Staff+	Prices the alternatives: "the Pareto plan is about 250k a year; provisioning for the worst case is nearly 5 billion. That settles it — we plan for the realistic peak and degrade under flash crowds. Egress is the line item I'd watch, which is why the CDN is load-bearing here."

Next: the two resources that actually constrain a service like this.