The Cost of Servers

Why this matters: an estimate in servers is abstract. The same estimate in money is a decision — and the gap between the plausible and pathological peak scenarios turns out to be the difference between a rounding error and five billion a year.

Key takeaway

Attach a price to your server count. In real projects, budget constraints are strict requirements — as binding as latency or availability, and far more likely to end a design.

The reference instance

To estimate cost, take an AWS EC2 m7i.16xlarge — a 64-core, 256 GB machine on a 4th-generation Intel Xeon, matching the reference server from Lesson 2:

Instance size	vCPU	Memory (GiB)	Instance storage	Network bandwidth	EBS bandwidth
m7i.16xlarge	64	256	EBS-only	25 Gbps	20 Gbps

Hourly cost with a one-year contract: 3.54816 USD per hour.

What each scenario costs

Applying that rate to the three server counts from the previous lesson:

Scenario	Servers	Cost per hour	Cost per year
Lower bound (uniform traffic)	2	7.10 USD	~62,000 USD
Pareto 80/20 assumption	8	28.38 USD	~249,000 USD
Peak load (all at once)	157,000	557,061 USD	~4.88 billion USD

2       * 3.548 =       7.096 USD/hour
8       * 3.548 =      28.38  USD/hour
157,000 * 3.548 = 557,061     USD/hour

Costs escalate quickly in the peak load scenario.

Cost as a design force

Money behaves exactly like the NFRs from the previous chapter: it constrains architecture, and pretending otherwise produces designs that never ship.

Cost driver	What inflates it	The lever
Compute	Provisioning for peak rather than average	Autoscale, use spot capacity for batch, degrade under extremes
Storage	Retaining everything forever at full replication	Tier cold data to cheaper classes; set retention policies
Egress	Serving bytes from origin	CDN — usually the single largest saving available
Cross-AZ / cross-region traffic	Chatty services spread across zones	Co-locate communicating services; batch calls
Idle redundancy	Hot standby for everything	Match failover mode to the actual availability target

A caveat on cloud pricing

These figures use on-demand-style pricing with a one-year commitment. Real bills differ, sometimes substantially:

Reserved capacity and savings plans cut compute meaningfully for predictable baseline load.
Spot instances are far cheaper for interruptible work — batch jobs, transcoding, analytics.
Volume discounts are routine at scale; large organizations do not pay list price.
Egress is frequently the surprise line item, and it is not in the instance price at all.

For a BOTEC, list price is the right choice: it is public, defensible, and conservative. Note the caveat, then use it.

Key takeaway

Convert servers to money before you defend a design. The cost delta between assumptions is often larger than the technical difference between architectures — and it is the number a business will actually make a decision on.

Interview signal by level

Level	What a strong answer sounds like
L4	Never mentions cost.
L5	Acknowledges it: "we should use reserved instances since the baseline load is predictable."
Staff+	Prices the alternatives: "the Pareto plan is about 250k a year; provisioning for the worst case is nearly 5 billion. That settles it — we plan for the realistic peak and degrade under flash crowds. Egress is the line item I'd watch, which is why the CDN is load-bearing here."

Next: the two resources that actually constrain a service like this.