Data Collection: Pull vs. Push

In one line: this is the decision an interviewer will probe hardest, because it determines who controls the data rate — and therefore whether your monitoring system can take down the infrastructure it monitors.

How pull works

The collector decides when to fetch and how often. Targets simply expose their current metrics and answer when asked.

A real-world example of this pull-based approach is DigitalOcean, which monitors millions of globally dispersed machines.

The drawbacks of push

Push inverts control: each application sends its own metrics to a central platform. That sounds simpler and creates four problems:

Drawback	Detail
Heavy traffic load	Each microservice sends its metrics to the monitoring system, producing heavy load on the infrastructure
Monitoring becomes a bottleneck	That load means monitoring can become a bottleneck for business operations
Network floods	Without appropriate care, continual push requests from all services can overwhelm the infrastructure, resulting in network floods
Operational burden	Daemons must be installed on each target to send metrics to the monitoring server — additional work per host

Push does have a real advantage worth stating: it can be near real time, because a service reports the moment something happens rather than waiting to be scraped.

Why push floods and pull doesn't

The difference is who controls the rate.

Under pull, the collector sets the scrape interval. If it is overloaded, it scrapes less often — the system degrades gracefully and the data rate is bounded by design.

Under push, every service decides independently. When something goes wrong, services typically emit more — more errors, more log lines, more metrics — so the monitoring traffic spikes at exactly the moment the infrastructure is already struggling. Monitoring load correlates with incidents, which is precisely the wrong correlation.

That is the same self-amplifying failure as retry storms from the Foundations module: a mechanism meant to help adds load exactly when the system can least absorb it.

The comparison

Aspect	Pull	Push
Who initiates	The monitoring system	Each application or server
Who controls the rate	The collector — bounded by design	Every target independently
Network congestion	Controlled	Risk of floods, worst during incidents
Freshness	Bounded by the scrape interval	Near real time
Target setup	Expose an endpoint	Install a daemon on every target
Knowing what to collect from	Needs service discovery	Targets announce themselves
Detecting a dead target	Scrape fails — absence is a signal	Silence is ambiguous: dead, or just quiet?

The last row is the underrated advantage of pull

Under pull, a target that does not answer produces a failed scrape — an explicit, timestamped event you can alert on. The absence of data becomes a positive signal.

Under push, a target that stops sending is indistinguishable from one that has nothing to report. You have to build a separate liveness mechanism to notice, which means push needs more machinery to detect exactly the failure monitoring exists to catch.

Recall from Lesson 1 that a rack losing power takes its agent with it. Pull notices that immediately; push has to infer it.

The messaging queue in between

The collector extracts relevant metrics from application logs via a distributed messaging queue. Each message carries service name, ID, and log description, which lets the collector identify and store specific metrics.

The queue decouples log production from metric extraction — applications write logs at whatever rate they like, and the collector consumes at whatever rate it can. That buffering is exactly the load-absorption property the Foundations module described for asynchronous messaging, applied here so a burst of log output cannot overwhelm the collector.

Key takeaway

Pull for control, push for freshness. The default is pull because it bounds the data rate and turns a missing target into a signal. The next lessons show where push earns its place anyway.

Interview signal by level

Level	What a strong answer sounds like
L4	"Services send their metrics to the monitoring system."
L5	Chooses pull with a reason: "pull, so the collector controls the rate and we don't flood the network with push traffic from every service."
Staff+	Names the correlation and the liveness point: "push load spikes during incidents, because failing services emit more — the same shape as a retry storm. And under pull a failed scrape is an explicit signal, whereas push silence is ambiguous, so push needs extra machinery to detect the very failures we care about."

Next: how the collector knows what to scrape.