An Agent Is a Loop

In one line: the difference between an assistant and a chatbot is that the assistant keeps going, and everything hard about it follows from that.

What changes

A single-turn system has a bounded shape. One model call, a known cost, a latency you can measure, and a failure that is visible immediately — the answer was wrong.

An agent decides how many steps to take. So:

Cost is unbounded until something bounds it. A task that takes three steps costs three model calls; the same task on a bad day takes forty. The variance between the median and the tail is enormous, and nothing in the model's objective limits it.

Latency is a sum of unknown length. Not "how fast is inference" but "how many inferences, plus the tool calls between them".

Failure is silent and slow. A single-turn system fails by giving a wrong answer. An agent fails by working — plausibly, expensively — on the wrong thing for twenty steps, and the first observable symptom is that nothing has happened yet.

Safety compounds. A wrong answer is a wrong answer. A wrong action changes state, and the next step reasons about a world it has already altered.

That third point is the one to internalise. An agent's characteristic failure is not an error — it is progress that is not progress.

The loop

Four stages, and the fourth is the one designs omit.

Plan. Decompose the goal into steps. Sometimes upfront, sometimes one step at a time.

Act. Take one step — usually a tool call.

Observe. Read the result and update what you believe about the world.

Check. Decide whether to continue, replan, or stop.

Most descriptions give the first three. The fourth is where termination lives, and without it the loop's stopping condition is "the model decided to stop", which is not a stopping condition — it is a hope.

Why termination is genuinely hard

The model has no reliable sense of whether it is making progress. It can call the same tool with the same arguments repeatedly, each time producing a fluent justification for why this attempt will work. Published work on agent failure modes names this directly: repetitive action subsequences that consume the interaction budget without changing anything.

Three distinct non-terminating shapes:

Exact repetition. The same tool, the same arguments, the same result. Easy to detect and the most common.

Cycling. A short sequence of steps repeating — search, read, search again with a trivially different query. Harder, because no individual step repeats.

Plausible drift. Each step is different and reasonable, and none moves toward the goal. This is the hardest to catch mechanically and the most expensive, because it looks exactly like work.

The three budgets

An agent needs limits it cannot argue with, and they are different from each other.

Steps. A hard cap on iterations. Crude, always necessary, and it fires on the tail rather than on the common case.

Cost. Tokens or currency, which is the one that actually protects the business. A step cap does not bound cost, because steps differ in size by orders of magnitude.

Wall clock. Because a user waiting is a different failure from a bill.

Exceeding a budget is not an error to hide. It is a result: I did not finish, here is where I got to, here is what I would do next. An agent that fails by going quiet is worse than one that fails by reporting.

What this means for the design

Three consequences that shape everything after.

The loop needs external control. The stopping logic, the budgets and the progress check live outside the model, in code that the model cannot talk its way past.

Every step is a checkpoint. The state after each step should be inspectable and resumable — because the user will interrupt, the process will crash, and a debugger needs somewhere to look.

Observability is per-step, not per-request. A trace showing one request that took ninety seconds is useless. A trace showing eleven steps, with the arguments and results of each, is where the answer is.

What it is not

Worth saying, because the reflex to build an agent is strong and often wrong.

If the task is one tool call, it is not an agent — it is a function call with a natural-language front end, and wrapping it in a loop adds cost, latency and a class of failure for nothing.

If the sequence of steps is known in advance, it is a workflow. Write it as one. A deterministic pipeline with an LLM at two of its stages is more reliable, cheaper and far easier to debug than an agent that rediscovers the sequence every time.

The loop earns its cost only when the path genuinely cannot be known upfront — when what to do next depends on what the previous step returned. That is the test, and most tasks that get built as agents fail it.

Next: scoping, and the autonomy question that decides everything.