What a Monitoring System Must Track
Why this matters: the Foundations module established that healthy nodes cannot tell what happened to a failed one. Monitoring is the machinery that answers that question — and its scope is far wider than "is the process running?"
Key takeaway
Automated monitoring identifies anomalies and notifies the alert manager or updates a dashboard. What it watches spans four layers most people never draw: software, hardware, power, and network.
The twelve signals
| Layer | What the system tracks |
|---|---|
| Process | Critical local process crashes · Resource usage anomalies (CPU, memory, disk, network) in specific processes |
| Server | Overall server health, including load averages and resource consumption · Hardware faults, such as memory failures or disk degradation · Connectivity to critical external services, like network file systems |
| Data center | Data center hardware status, including network switches and load balancers · Power consumption at server, rack, and data center levels · Power events affecting servers, racks, or the data center |
| Network and global | Routing information and DNS status · Network latency within and across data centers · Network status at peering points · Global service health across data centers, such as CDN performance |
Why the scope is this wide
Each layer fails in a way the layer above cannot detect on its own:
The last two rows are the important ones. A rack losing power takes the monitoring agent with it, so absence of data is the signal. A congested peering point produces no error anywhere in your fleet — every server reports itself healthy while users time out.
That second case is the gray failure from the Foundations module: the system's view of its own health disagrees with what users experience. Monitoring only catches it if it is watching the network path, not just the servers.
What the industry publishes
Cloud providers run exactly this and expose the output publicly:
| Provider | Status page |
|---|---|
| AWS | health.aws.amazon.com/health/status |
| Azure | status.azure.com/en-us/status |
| status.cloud.google.com |
These pages are the customer-facing surface of a monitoring system of exactly this shape — signals collected per region and service, aggregated, and rendered as health state.
Key takeaway
Monitor across process, server, data center, and network. The signals that are hardest to collect — power, peering, DNS — are the ones that catch failures where every individual server still reports itself healthy.
Interview signal by level
| Level | What a strong answer sounds like |
|---|---|
| L4 | "We'd monitor CPU, memory, and whether the service is up." |
| L5 | Covers the stack: "process health, server resources, hardware faults, plus network latency between data centers." |
| Staff+ | Reaches the invisible failures: "I'd also watch power at rack level and peering-point status, because those produce failures where every server reports healthy and users still time out. Absence of data from a rack is itself a signal." |
Next: the three components that collect and serve it.