Acting on a Moving Page

In one line: the agent decides what to click based on a page that has already changed by the time it clicks.

The gap

Every step has an interval between observation and action: the page is captured, the observation is built, a model call is made — which takes a second or more — and then the click is dispatched.

In that window the page can load remaining content, shift its layout, replace a section, or navigate itself. So the element at label 14 when the observation was taken may be a different element by the time the click lands.

This is the browsing-agent version of a stale plan from the previous chapter, at a much shorter timescale, and it is why waiting is a first-class concern rather than an implementation detail.

Waiting properly

The naive approach is a fixed sleep, and it is wrong in both directions — too short and the page is not ready, too long and every step pays for the worst case.

Better signals, roughly in order:

Network idle. No outstanding requests for a short interval. A decent general-purpose signal, and it is defeated by polling and long-lived connections, which are common.

Element presence. Wait for the specific thing you expect. Much more precise, and it requires knowing what to expect — which you do, because the action had an intention.

DOM stability. The tree has stopped changing for a short interval. A useful catch-all for single-page apps that mutate without network activity.

The practical answer is a combination with a bounded timeout: wait for the expected element, fall back to stability, and give up after a limit rather than hanging.

Re-observe before acting

The discipline that resolves the gap, and it is a genuine cost trade.

Between deciding and acting, re-capture the element and confirm it is still what it was — same role, same accessible name, same position within tolerance. If it changed, the observation is stale and the step should be re-planned rather than executed.

This costs a second capture per step, and it prevents the most confusing class of failure in these systems: the agent clicked the right label on the wrong element, and every downstream observation is now interpreted through a wrong belief.

A cheaper version that catches most of it: capture a small fingerprint of the chosen element at observation time — role, name, and rough position — and verify only that fingerprint before clicking. Much less than a full re-observation and it catches the case where the element was replaced.

Clicks are not idempotent

The property that makes retries dangerous here, and it is worse than in an API-driven agent.

A click that appears to fail may have succeeded. The response was slow, the page had not updated, the agent retried — and now the item is in the basket twice, or the form submitted twice.

Three mitigations.

Verify before retrying. Never retry a click without re-observing to see whether it worked. This is the same discipline as verification generally, applied to the retry path specifically.

Prefer navigation over repetition. If a step failed, going back to a known state and re-approaching is safer than clicking again from an unknown one.

Treat submits as unrepeatable. For genuinely consequential actions, a failure should escalate rather than retry. The previous chapter's rule — the last irreversible click is where the human belongs — makes this mostly moot for transactional flows.

What else moves

Three specific page behaviours that break naive agents, each with a cheap handling.

Overlays and modals. A cookie banner, a newsletter prompt, an age gate. They intercept clicks on elements that are visibly present, producing a click that hits the overlay instead. Detect a modal in the observation and dismiss it as an explicit first step rather than clicking through it.

Infinite scroll. Content the agent needs is not in the DOM until it scrolls. So "not found" is ambiguous between absent and not-yet-loaded, and the agent needs a bounded scroll-and-search rather than concluding after one look.

Navigation the agent did not cause. A redirect, an interstitial, a session timeout. The page is now somewhere unexpected, and the agent's plan assumes otherwise. Checking the URL as part of every observation is nearly free and catches this immediately.

That last one is worth building in from the start: the URL is part of the observation. An agent that does not notice it has been redirected to a login page will keep trying to click things that no longer exist.

Session state

Briefly, because it interacts with everything above.

The browser accumulates state — cookies, storage, history. That is what makes multi-step flows possible, and it means a failed task can leave the session in a state the next task inherits if the context is reused.

Which is the argument for isolated contexts from the scoping lesson, and for one more thing: a task that fails partway should record what it did, so the state it left behind is known rather than discovered later.

Next: confirming the action did what it was supposed to.