Confidence, Escalation, and Feedback

In one line: escalation is the system's safety valve, and it is triggered by a number the design never defines. The feedback loop meant to improve quality measures something adjacent to quality.

The confidence problem

The human escalation module uses confidence scoring to detect when the LLM cannot adequately resolve a query and routes the conversation to a live agent.

Attention: Confidence scoring thresholds require careful calibration. Setting the threshold too low floods agents with unnecessary escalations. Setting it too high means frustrated users never reach a human.

The warning is right and the chapter never says what is being thresholded

Calibration advice is useful only if you know what the number is. And the intuitive answer — ask the model how confident it is, or read its token probabilities — does not work.

LLMs are not calibrated. A model's token log-probabilities measure how likely that phrasing is given its training, not how likely the claim is to be true:

"Your refund will be processed within 5 business days."
  -> a fluent, high-probability sentence
  -> equally high-probability whether or not it is your actual policy

A model is most confident precisely when it is fluently wrong, because fluency and probability are the same quantity. Asking it "how confident are you?" is worse still — the answer is generated text, subject to the same failure.

So a usable confidence signal has to come from somewhere other than the generation itself. The reliable ones, roughly in order of value:

Retrieval score. Did the vector search return anything genuinely relevant? If the best chunk scores below threshold, the model is about to answer from parametric memory (Lesson 5) — and this is the strongest available signal, because it is measured before generation.

Grounding check. With citations (Lesson 5), you can verify each claim maps to a retrieved chunk. Uncited claims are ungrounded generation.

Self-consistency. Sample the answer two or three times; if they disagree materially, the model is uncertain. Reliable and expensive — it multiplies the cost of exactly the queries you were unsure about.

A separate classifier, trained on historical escalations, taking as features the intent, the retrieval scores, the conversation length, and sentiment. Actually calibrated, because it is trained on outcomes.

Explicit signals. The user asked for a human; sentiment turned negative; the same question has been asked twice; the intent is on a mandatory-escalation list.

Confidence in an LLM system is assembled from signals around the model, not read out of it. That is the answer to what is being thresholded.

The strongest signals are available before generation, which the workflow does not exploit

Lesson 4 noted that the design checks confidence at step 9, after generation and after moderation. Now the cost of that ordering is clear.

Retrieval returns nothing relevant     -> known BEFORE generating
User explicitly asks for a human       -> known BEFORE generating
Intent is 'dispute charge'             -> known at NLU, BEFORE generating
User expressed frustration             -> known BEFORE generating
Answer is internally inconsistent      -> known only AFTER generating

Four of five are available upstream, and they are the more reliable four.

Escalating early saves the generation cost on the most expensive tier, and — more importantly — it saves the user from reading a hedged non-answer before being handed off. A confident-sounding evasion followed by "let me connect you to an agent" is worse than an immediate handoff, because it wastes the user's time and damages trust in every answer the bot gave earlier.

Escalate on the earliest sufficient signal, and reserve post-generation checks for what only generation can reveal.

Escalation

Detect when confidence is too low and transfer the conversation to a human agent while preserving the full conversation context.

Context preservation is what makes escalation a handoff rather than a restart

This is the requirement that separates a good escalation from the experience everyone has had: repeating your entire problem to a second party.

What the agent needs is more than the transcript:

The conversation history               <- from Redis
The intent and entities NLU extracted  <- order 4471, "billing dispute"
What the bot already tried             <- which actions it called
What it retrieved                      <- so the agent sees the same policy
Why it escalated                       <- the reason parameter
The customer's account state           <- from function calls already made

The design carries escalateToHuman(session_id, reason), and session_id is the key to all of it — which is the payoff of Lesson 4's observation that the session is the system's only per-request state.

The reason parameter is the underrated one. "low_confidence" versus "complex_query" versus "user_requested" tells the agent how to open, and in aggregate it is the best available signal about where the bot is weak — which intents escalate most, which retrieval gaps recur.

An escalation is a handoff, and a handoff without context is a restart. The context transfer is also what makes the design's own warning tractable: if escalations are cheap and smooth, you can afford a lower threshold.

Escalation capacity is finite, and the threshold is really a capacity decision

The source's framing — too low floods agents, too high strands users — is right and slightly incomplete. Agent capacity is fixed in the short run, so the threshold is not only a quality knob:

250M queries/day, 1% escalation rate  = 2.5M escalations/day
                  0.1%                = 250K escalations/day

At a few hundred conversations per agent per day, those imply wildly different support organizations. The threshold is a staffing decision wearing a quality costume.

Two consequences the design does not draw:

Escalation needs a queue with priority, not just a transfer. The design routes "via pub-sub", which is the right transport, and nothing describes what happens when agents are saturated. A customer disputing a charge should outrank one asking about store hours.

The threshold should be adaptive. When the agent queue is short, escalate readily; when it is long, raise the bar and let the bot attempt more. That is load-shedding applied to a human resource — and it beats a fixed threshold that is wrong at both ends of the day.

When a fallback path has fixed capacity, the threshold that routes to it is a capacity control, and it should respond to load.

Feedback

submitFeedback(): Collects user ratings and feedback on responses to improve system performance over time. This data is used for monitoring, retraining, and quality evaluation.

Thumbs-up measures satisfaction, and satisfaction is not correctness

The gap is specific and it is the reason naive feedback loops make systems worse.

Users rate confident, fluent, agreeable answers highly — including when they are wrong, because the user usually cannot tell. The failure only surfaces later, when they act on it:

Bot: "Yes, you can return that after 60 days."   (policy says 30)
User: thumbs up — this is the answer they wanted
...three weeks later: a rejected return and an angry call

And it fails in the other direction too: a correct answer the user did not want to hear — "that item isn't eligible for a refund" — reliably earns a thumbs-down.

Thumbs-up correlates with: fluency · agreeableness · confidence
Thumbs-up does NOT measure: factual accuracy

Optimizing on that signal produces a bot that is more agreeable and less accurate — which is a well-documented failure mode of reinforcement from human preference, arriving here through a much simpler mechanism.

Better signals, all of which this system can already collect:

Did the user escalate afterwards?     <- strong NEGATIVE signal
Did they rephrase and ask again?      <- the answer didn't land
Did the conversation end quickly?     <- likely resolved
Did they contact support again in 48h? <- the bot did NOT resolve it
Did a function call succeed?          <- objective
Agent disposition after escalation    <- a human's verdict on the bot's attempt

Behavioural signals beat stated satisfaction, because they measure whether the problem was solved rather than whether the answer was pleasant. Resolution rate — not thumbs-up rate — is the metric this system exists to move.

For a RAG system, the fix for a bad answer is usually the knowledge base, not the model

The design says feedback feeds "retraining." For a retrieval-augmented system that is usually the wrong response, and it is expensive.

When an answer is wrong, work back through Lesson 5's failure modes:

Only the last row calls for retraining, and it is the rarest. That is the whole point of RAG from Lesson 1: knowledge lives in the vector database precisely so it can be corrected without touching weights.

Which is why citations matter operationally and not just for user trust — they tell you which row you are in. Without them, every bad answer looks like a model problem.

In a RAG system, most quality regressions are content or retrieval regressions, and reaching for retraining first is expensive and slow.

'Model drift in 30 to 90 days' conflates two different things

Educative byte: LLM performance can degrade within 30 to 90 days due to model drift. Without continuous monitoring and periodic retraining, response quality can silently erode.

The observation that quality erodes is right; the mechanism is misattributed. A deployed model's weights do not change, so the model does not drift. Three other things do:

KNOWLEDGE STALENESS  products, policies, prices change
                     -> fixed by the INGESTION PIPELINE, not retraining
                     -> this is the dominant cause, and RAG already handles it

QUERY DRIFT          users ask about new products, new issues, new phrasings
                     -> fixed by adding CONTENT and retuning retrieval

WORLD DRIFT          the environment the model reasons about changes
                     -> the slowest, and the only one retraining addresses

The distinction is the same one that building block drew between skew and drift: one is an engineering defect and one is a fact about the world.

Getting it wrong is costly: retraining a model whose real problem is a stale knowledge base spends weeks and fixes nothing, because the new model will have the same outdated documents injected into its prompts.

For a RAG system, "drift" is mostly knowledge staleness, and the ingestion pipeline is the answer — which is precisely what RAG was adopted for.

Conversation history grows every turn, and nothing bounds it

One gap the design leaves entirely. Redis holds session history and it is "injected into each prompt" — so:

Turn 1:  prompt = instructions + chunks + query          ~2,200 tokens
Turn 5:  prompt = instructions + chunks + 4 prior turns  ~4,500 tokens
Turn 20: prompt = instructions + chunks + 19 prior turns ~12,000 tokens

Every turn re-sends everything before it, so prompt length grows linearly and cumulative cost grows quadratically in conversation length. Lesson 3 established that prompt length drives time-to-first-token, so a long conversation gets slower and more expensive simultaneously — exactly when the user is already frustrated.

Eventually it hits the context window and truncates, usually silently and from the wrong end.

Three standard bounds:

Sliding window — keep the last N turns verbatim. Simple, and it loses the order number from turn one.

Summarize and compact — periodically replace older turns with a short summary of the established facts. Preserves what matters at a fraction of the tokens.

Extract structured state — keep a small record (order_id, issue_type, actions_taken) alongside a short recent window. Most of what "context" means in support is a handful of facts, not a transcript.

The third is best here, and prompt caching (Lesson 6) makes the growth cheaper without bounding it — the shared prefix is prefilled once per session, but the tokens still count against the window and against per-token cost.

Conversation memory must be bounded and compacted, or long sessions degrade in latency, cost, and quality at once.

Next: the evaluation.