The Agent and Collector Design
Why this matters: this is the design that actually solves the problem. It inverts probing — instead of you sampling the network, the users tell you — which fixes the coverage ceiling by construction.
Key takeaway
Instead of external vantage points, embed monitoring directly into the client application: an agent that sends reports when failures occur, and a collector that receives them and must be independent of the primary service.
The two components
| Component | Responsibility |
|---|---|
| Agent | A module embedded in the client application that sends reports when failures occur |
| Collector | A standalone service that receives error reports. Independent of the primary service, so it remains reachable even when the main service fails. Reports are analyzed to detect spikes in client-side errors |
The essential move is in that diagram: the client cannot reach the service, and it reports that fact somewhere else. If the report went to the primary service it would fail for exactly the same reason as the original request — which is why the collector's independence is not a nice-to-have but the entire premise.
Scaling the collector tier
Collectors are organized in a hierarchical aggregation architecture. Two supporting decisions:
| Decision | Detail |
|---|---|
| Deploy close to client networks | Reduces latency — reports arrive from a nearby collector rather than crossing the world |
| Stream processing | Enables near real-time analysis, so a spike is visible in seconds rather than after a batch job |
Sampling
To control costs, the system may sample reports — for example collecting data from 1% of users — instead of capturing every event. A system that ingests and processes every event would require substantially more infrastructure and cost.
The three questions this raises
Embedding a reporter in the client creates three problems the rest of the chapter addresses:
| Question | Covered in |
|---|---|
| Can a user activate and deactivate client-side reports? | Lesson 5 |
| How do agents reach collectors under faulty conditions? | Lesson 6 |
| How do we protect user privacy? | Lesson 7 |
Note that two of the three are about the user, not about the system. That ratio is unusual among the building blocks in this course, and it is the defining characteristic of client-side monitoring: you are running code on someone else's device.
Key takeaway
Agent plus independent collector solves the coverage problem by reporting from where users actually are. The collector's independence from the primary service is the load-bearing constraint — everything else is aggregation and cost control.
Interview signal by level
| Level | What a strong answer sounds like |
|---|---|
| L4 | "The client could report errors back to us." |
| L5 | Gets the independence right: "an agent in the client reporting to a separate collector — separate, because if it reported to the main service the report would fail too." |
| Staff+ | Adds coverage and cost reasoning: "this fixes probing's coverage ceiling — reports come from every network that has a user in it. I'd aggregate hierarchically with collectors near clients, stream-process for near real-time spike detection, and sample at around 1%, because we're estimating a rate rather than reconstructing sessions." |
Next: whether the user agreed to any of this.