Data Collection: Pull vs. Push
Why this matters: this is the decision an interviewer will probe hardest, because it determines who controls the data rate — and therefore whether your monitoring system can take down the infrastructure it monitors.
Key takeaway
The data collector gathers metrics from services across data centers using a pull strategy. It extracts relevant metrics from application logs via a distributed messaging queue, where each message contains the service name, ID, and a log description.
How pull works
The collector decides when to fetch and how often. Targets simply expose their current metrics and answer when asked.
A real-world example of this pull-based approach is DigitalOcean, which monitors millions of globally dispersed machines.
The drawbacks of push
Push inverts control: each application sends its own metrics to a central platform. That sounds simpler and creates four problems:
| Drawback | Detail |
|---|---|
| Heavy traffic load | Each microservice sends its metrics to the monitoring system, producing heavy load on the infrastructure |
| Monitoring becomes a bottleneck | That load means monitoring can become a bottleneck for business operations |
| Network floods | Without appropriate care, continual push requests from all services can overwhelm the infrastructure, resulting in network floods |
| Operational burden | Daemons must be installed on each target to send metrics to the monitoring server — additional work per host |
Push does have a real advantage worth stating: it can be near real time, because a service reports the moment something happens rather than waiting to be scraped.
The comparison
| Aspect | Pull | Push |
|---|---|---|
| Who initiates | The monitoring system | Each application or server |
| Who controls the rate | The collector — bounded by design | Every target independently |
| Network congestion | Controlled | Risk of floods, worst during incidents |
| Freshness | Bounded by the scrape interval | Near real time |
| Target setup | Expose an endpoint | Install a daemon on every target |
| Knowing what to collect from | Needs service discovery | Targets announce themselves |
| Detecting a dead target | Scrape fails — absence is a signal | Silence is ambiguous: dead, or just quiet? |
The messaging queue in between
The collector extracts relevant metrics from application logs via a distributed messaging queue. Each message carries service name, ID, and log description, which lets the collector identify and store specific metrics.
The queue decouples log production from metric extraction — applications write logs at whatever rate they like, and the collector consumes at whatever rate it can. That buffering is exactly the load-absorption property the Foundations module described for asynchronous messaging, applied here so a burst of log output cannot overwhelm the collector.
Key takeaway
Pull for control, push for freshness. The default is pull because it bounds the data rate and turns a missing target into a signal. The next lessons show where push earns its place anyway.
Interview signal by level
| Level | What a strong answer sounds like |
|---|---|
| L4 | "Services send their metrics to the monitoring system." |
| L5 | Chooses pull with a reason: "pull, so the collector controls the rate and we don't flood the network with push traffic from every service." |
| Staff+ | Names the correlation and the liveness point: "push load spikes during incidents, because failing services emit more — the same shape as a retry storm. And under pull a failed scrape is an explicit signal, whereas push silence is ambiguous, so push needs extra machinery to detect the very failures we care about." |
Next: how the collector knows what to scrape.