Scoping, and the Limits You Respect

In one line: decide whether the agent reads or acts, then decide which sites it is welcome on, because the second question is not optional.

Read or act

The scoping split that changes everything.

Read-only browsing — find information, compare options, summarise. Failures are cheap: a wrong answer the user can check. It can visit many sites, and it needs no permission model beyond the user's own.

Transactional browsing — fill forms, submit, book, purchase, change settings. Failures are expensive and often irreversible, and the agent is now acting as the user against a third party.

Nearly every hard question in this chapter is easier if the agent is read-only, and a design that quietly assumes transactional capability has taken on the whole problem without saying so.

The autonomy ladder from the previous chapter applies directly: read freely, draft the transaction, and let the human submit — which for browsing has a pleasant property, since the last click is often the only irreversible one. An agent that fills the form and stops before Submit has done nearly all the work and none of the damage.

Whose infrastructure

The part that is genuinely a design constraint rather than a legal footnote.

An agent browsing a site is consuming somebody else's servers, and it is doing so faster and more persistently than a person would. That has three implications a design should state.

robots.txt and terms of service are inputs. Some sites permit automated access, some restrict it, some prohibit it. Reading and honouring that is table stakes, and a system that ignores it will eventually be blocked — which is a reliability problem before it is anything else.

Rate limits belong to them. The same point as enterprise search connectors: an aggressive agent degrades the site for its actual users, and the correct response to a rate limit is to slow down rather than to work around it.

Identify yourself. A descriptive user agent with a contact route is what lets a site operator talk to you instead of blocking you. Anonymous high-volume traffic gets treated as abuse, correctly.

When the site says no

Bot detection and CAPTCHAs are a signal, and the design decision is what to do with it.

The honest treatment: a CAPTCHA means the site is asking for a human, and the correct response is to get one or to stop. Hand the session to the user to complete, or fail the task with a clear explanation. Attempting to defeat it is both outside what these systems should do and a losing arms race that produces an unreliable product.

Which makes the design consequence concrete. The agent needs:

A handoff path. Surface the live session to the user, let them complete the challenge, and resume. That requires the durable state from the previous chapter and a way to show the browser.

A clean failure. "This site requires a human check that I can't complete" is a good outcome. Silently retrying is not.

A record. Sites that consistently block automated access belong on a list the planner consults, so the agent stops choosing them.

Authentication

The adjacent question, and it deserves care because it involves the user's credentials.

Three approaches, in order of preference.

Delegated auth — OAuth or a similar grant, where the agent gets a scoped token and never sees a password. Correct wherever available.

A session the user established. The user logs in themselves, and the agent operates in that authenticated context. No credential ever reaches the agent, and the session is revocable.

Stored credentials. The agent holds the user's password and types it. Avoid this. It is a serious liability, it breaks with any second factor, and it makes the agent's compromise equivalent to the user's.

A design that reaches for the third option without discussing the first two has under-thought the most sensitive part of the system.

Isolation

One infrastructure decision worth naming, because it protects the user rather than the system.

Each task runs in a fresh, isolated browser context. Not a shared browser, not a shared profile.

Three reasons. Session bleed — one user's cookies must never be visible to another's task. Contamination — a task that ended up somewhere unpleasant should not leave state behind. And reproducibility — a task that starts from a known-clean context behaves the same way twice, which matters enormously for debugging.

The cost is that every task pays a cold start, and any authenticated session has to be re-established. That is a real overhead and it is the right trade.

Committing

A research and transaction agent for a consumer product. Read-only browsing across many sites; transactional work only on sites where the user has an established session, and always stopping before the final irreversible submit. robots.txt and rate limits are honoured, the agent identifies itself with a contact route, and a site that blocks automated access is recorded and avoided rather than worked around. Each task runs in an isolated browser context. Credentials are never stored — the user authenticates and the agent inherits the session.

Next: what the agent actually sees.