Feedback and Billing

In one line: these are the two paths deliberately kept off the critical path, and both are behind pub-sub for the same reason. They also share a subtler property: both measure something the request itself cannot know.

Billing

How does the payment system determine how much to charge a user for a request in a system like ChatGPT, especially when billing is based on token usage or premium features?

And answers it:

The payment system calculates charges based on usage logs/data, which track details like the number of tokens used, the complexity of the request, and any premium features accessed.

After processing the request, the billing service computes the appropriate cost based on factors like token count or other pricing metrics. This information is then sent to the payment system for user approval to ensure the user is fully informed and consents to the charge.

Billing must be post-hoc, and the reason is structural

You cannot price the request in advance, because the cost depends on the output length and the output length is not known until it exists.

This is genuinely unusual. Almost every other billable operation in computing has a knowable cost at request time — a storage write is priced by bytes you already have, an API call is priced per call. Here a prompt might generate 20 tokens or 2,000, a hundredfold difference, and nothing in the request predicts which.

So metering runs after generation, over three quantities:

Input tokens — the assembled prompt, including system instructions, history, and retrieved context.
Output tokens — what was generated. Usually priced several times higher, which Lesson 4 explains: output tokens each require a full sequential forward pass, while input tokens are processed in one parallel prefill.
Model and features — a large model costs more per token than a small one.

That input/output price asymmetry is not a pricing whim. It reflects the actual cost structure of the hardware, and being able to explain it from the prefill-versus-decode distinction is a strong signal that you understand what you are billing for.

"User approval" before finalizing does not work per request, and the fix is standard

Read literally that is unworkable. Approving each of 1.5 billion daily charges individually would make the product unusable, and the response has already been delivered — refusing after the fact does not undo the compute.

What real systems do instead is the same two-phase pattern card networks use:

Before generating: check a prepaid balance or quota. Cheap, synchronous, milliseconds. If exhausted, refuse before spending GPU time — the same "avoid invoking the model" saving as input moderation and semantic caching.

After generating: meter and settle against the actual token count.

The "approval" the design describes belongs at the boundaries, not per request — consenting to a pricing tier, approving a top-up, being warned before crossing a spend threshold.

The gap between the two phases is a real revenue risk: a user can consume more than their balance within a single long generation, or events can be lost between the application server and the billing service. That is bounded by capping maximum output length per request, and by making the usage stream durable with reconciliation rather than assuming delivery.

One quantity, three consumers

Note what the usage stream feeds:

Consumer	Uses tokens for
Billing service	What to charge
Rate limiter	Whether to allow the next request
Capacity planning	How many GPUs to buy

Tokens are simultaneously the unit of work, the unit of cost, and the unit of price. That alignment is unusual and it is worth naming, because it means these three systems can share one measurement pipeline instead of each inventing a proxy.

Compare the awkwardness elsewhere in the course — storage systems bill by gigabyte-months while being constrained by IOPS, or CDNs bill by egress while being constrained by cache capacity. Those mismatches cause the pathologies where the cheapest usage pattern is the most expensive to serve. Here, the thing you charge for is the thing that costs you, which makes the incentives line up naturally.

Feedback

If the user provides feedback, it is collected via the feedback module and stored in the feedback database. Structured feedback (ratings, thumbs up/down) is stored in a relational database. Unstructured feedback (comments, session metadata) is stored in a document database. This data helps improve the system over time through analysis and model updates.

The path from thumbs-down to better model

The requirement says feedback should "improve accuracy and performance over time." The mechanism is worth spelling out, because "we collect feedback" is where most answers stop.

Two things about this loop deserve emphasis.

It closes in weeks, not seconds. The dotted line is the point. The user who pressed thumbs-down receives no benefit from having done so; the benefit accrues to future users after a retraining cycle. That is why the whole path is asynchronous, and why treating it as part of the request flow — as the high-level diagram in Lesson 6 does — is misleading.

The evaluation gate is not optional. A new model trained on feedback can be worse overall while being better on the specific complaints. Without a held-out evaluation suite you have no way to know, and you will ship regressions. That gate is the only quality control in the entire design, and the design never mentions it.

Feedback is heavily biased, and using it naively degrades the model

Treating the feedback stream as ground truth is the most likely way to make this loop actively harmful.

Very few users rate anything. Perhaps a few percent, and they are not a random sample.

Negative feedback dominates. People react when something is wrong far more than when it is right, so the raw distribution overstates failure and, worse, overstates certain kinds of failure.

Users rate satisfaction, not correctness. This is the sharpest problem. A confident, fluent, well-formatted answer that is factually wrong often gets a thumbs-up — the user cannot tell. A correct answer that says "I don't know" or contradicts what the user believed often gets a thumbs-down.

Train directly on that signal and you optimize for plausibility rather than truth. The model learns to be more confident, more agreeable, and more polished — which is precisely the wrong direction given that Lesson 2 established the system has no accuracy safeguard at all.

So the feedback path needs correction rather than raw ingestion: weight against known rater bias, use paired preferences ("which of these two responses is better") rather than absolute ratings, sample and label traffic with expert reviewers instead of relying only on volunteers, and hold out an evaluation set the feedback loop cannot influence.

The general lesson transfers well beyond this system: an optimization loop drives whatever it measures. If the measurement is user satisfaction, you get a system optimized for satisfaction — and satisfaction and correctness diverge exactly where correctness matters most.

Implicit feedback is larger and less biased than explicit

Thumbs are the visible signal but by far the smaller one. Behaviour is available on every request:

Did the user regenerate the response? A strong negative.
Did they rephrase and ask again? The answer missed.
Did they copy the output? A strong positive, especially for code.
Did the conversation continue naturally, or stop abruptly?
How long did they stay on the response?

These cover essentially all traffic rather than a few percent, and they are less prone to the rating-selection bias above.

They come with their own confound — regeneration might mean the answer was wrong, or that the user wanted a different tone — so they are noisier per event and much stronger in aggregate.

Mentioning implicit signals unprompted is a good marker of having thought past the diagram, since the design only describes the explicit path.

Key takeaway

Billing is necessarily post-hoc because output length is unknown in advance, which is unusual — so the workable pattern is check a balance before generating, meter after, with output tokens priced above input tokens because decode is sequential and prefill is parallel. Tokens serve billing, rate limiting, and capacity planning at once, so what you charge for is what costs you. Feedback closes in weeks, and its evaluation gate is the only quality control in the design. Most importantly, feedback is biased toward satisfaction rather than correctness — training on it naively optimizes for confident plausibility, which is the opposite of what a system with no accuracy safeguard needs. Implicit signals — regeneration, rephrasing, copying — cover all traffic rather than a few percent.

Next: evaluating the design against its requirements.