Free preview

The BGP Leak: An Outage You Cannot See

Why this matters: this is the concrete case that justifies the whole chapter. It is a total outage for affected users, produced entirely outside your infrastructure, and invisible to every metric you collect.

Key takeaway

A peer ISP mistakenly announced incorrect BGP routes for a major service provider. The leak redirected customer traffic through unintended networks, preventing it from reaching its destination. Users could not access the service, and the provider was not immediately aware — because the outage originated outside its own infrastructure.

What happened

Read the two ends of that diagram together. On the left, users are completely unable to reach the service. On the right, every dashboard is green — correctly, because nothing the provider owns is broken.

Not an isolated incident

BGP route leaks are not isolated incidents. On April 16, 2021, an autonomous system mistakenly announced more than 30,000 BGP prefixes, resulting in a 13x increase in inbound traffic. Network operators detected the surge and mitigated the misconfiguration.

DetailValue
DateApril 16, 2021
Scale of the mistakeMore than 30,000 BGP prefixes announced in error
Effect13x increase in inbound traffic to the affected network
How it endedNetwork operators detected the surge and mitigated it

Why traditional monitoring misses it

Traditional infrastructure monitoring may not capture these external routing events, so engineers must implement dedicated external connectivity monitoring.

The reason is structural, and worth stating plainly:

Monitoring typeVantage pointSees a BGP leak?
Server metricsInside your infrastructure✗ — everything is genuinely healthy
Application logsInside your infrastructure✗ — the requests never arrive to be logged
Load balancer health checksInside your infrastructure✗ — backends respond normally
Traffic volumeInside your infrastructureweak — a dip, but inside normal variance if regional
External connectivity monitoringOutside, from the client's position

Every row that fails shares one property: the vantage point is inside. You cannot detect a failure in the path to your building by looking around inside the building.

Key takeaway

A BGP leak is a total outage for affected users, caused by a third party, with every one of your own signals reporting health. It is the clearest possible argument that some monitoring must live outside the system it watches.

Interview signal by level

LevelWhat a strong answer sounds like
L4"Network problems could stop users reaching us."
L5Names the mechanism: "a BGP misconfiguration by a peer can redirect our traffic, and we'd have no internal signal because our servers are fine."
Staff+Generalizes it: "every internal signal is blind here by construction — you can't see a failure in front of your infrastructure from inside it. Same principle as failure independence for replicas: to detect a failure in a layer, you have to observe from outside that layer."

Next: the obvious solution, and where it runs out.

Enjoying the preview?

Create a free account to unlock the rest of this course, the in-browser judge, and live AI mock interviews.

Sign up free to continue