Scoping, and the Cost Matrix

In one line: "fraud detection" names several different systems, and the one thing they share is that you cannot pick a threshold until someone tells you what each mistake costs.

Which fraud

These need different systems. Payment fraud is a synchronous decision on a single event. Account takeover is a change detection problem — the question is not whether this behaviour is unusual in general but whether it is unusual for this account. Synthetic identity is the hardest, because there is no victim to dispute anything and the account may behave perfectly for a year before busting out.

Promo abuse is worth a note because it breaks the usual framing: each individual signup is legitimate, and only the pattern across accounts is abusive. That is a graph problem from the start, not a per-transaction classification.

Whose money

The question that decides how aggressive to be, and candidates skip it.

The platform bears the loss — a marketplace covering chargebacks, a bank refunding a customer. Every pound of fraud is a pound off the bottom line, so blocking is directly valuable.

A third party bears it — the issuing bank absorbs some card fraud, an insurer covers the rest. Your incentive to block is weaker and mostly reputational or contractual.

The user bears it — a peer-to-peer transfer they authorised under deception. Now the fraud is not a transaction you can decline: the customer wants it to go through, and stopping it means overriding someone's explicit instruction.

That last case is genuinely different and worth raising. An authorised push payment fraud has a legitimate account holder, a correct password, a valid device and full intent. Every signal says approve, and the customer is being deceived by someone else. The intervention is not a decline; it is a warning, a delay, or a question.

The cost matrix

Four numbers, and until they exist there is no correct threshold.

                    Actually fraud       Actually legitimate
  Approved          LOSS: transaction    correct — revenue
                    + chargeback fee
                    + processing cost
  Declined          correct — avoided    COST: lost sale
                                         + lost customer lifetime
                                         + support contact

Two observations that reframe the design.

The false-negative cost is more than the transaction. A chargeback carries a fee, an operational cost to process, and — beyond a threshold ratio — the risk of penalties from the card networks. So the loss is a multiple of the amount, not the amount.

The false-positive cost is much larger than the sale. Someone declined at checkout does not merely fail to buy this thing. A meaningful fraction never return, so the cost includes their remaining lifetime value. It is also a support contact, and it lands hardest on exactly the high-value customers whose unusual purchases look most like fraud.

The threshold falls out

With the four costs, the operating point is arithmetic rather than judgement.

Approve when the expected cost of approving is lower than the expected cost of declining:

p(fraud) x cost_of_fraud_loss   <   (1 - p(fraud)) x cost_of_wrong_decline

Rearranged, that gives a threshold on p(fraud) set entirely by the ratio of the two costs.

Which produces the sentence worth having ready: the model's job is to estimate the probability; the business's job is to say what the errors cost; the threshold is where those meet. A data scientist choosing a threshold alone is making a commercial decision without the inputs.

And it means the threshold varies. A £10 digital purchase and a £4,000 electronics order have different loss exposures, so they should not share an operating point — the threshold is a function of amount, not a constant.

The other constraint: review capacity

The practical limit that turns theory into operations.

Many fraud systems send borderline cases to human analysts. That queue has a fixed throughput — a team can review some number of cases per day, and no more.

So the operating point is often not set by the cost matrix at all. It is set by capacity: choose the threshold that fills the review queue and no more, because a case that queues for three days has already lost the customer.

That is worth stating because it changes what "improving the model" means. If the queue is the binding constraint, a better model does not let you review more cases — it lets you review better-chosen cases, and the metric that matters is precision at the fixed review volume rather than anything computed over the whole distribution.

Scale, and what it decides

Three numbers.

Transaction rate sets the serving budget. Payment authorisation typically has a few hundred milliseconds end to end, of which fraud scoring gets a fraction.

Fraud rate is usually well under one percent, which makes accuracy meaningless and every positive label precious.

Average value and its distribution matter more than the mean. Fraud concentrates in the tail, so a system optimising average loss can be perfect on the bulk of transactions and lose most of its money on a handful.

Committing

Card-not-present payment fraud on a marketplace. We bear the chargeback, so blocking is directly valuable. Roughly two thousand transactions per second at peak, fraud around 0.3% of attempts and a larger share of value. Fifty milliseconds for the fraud decision inside a three-hundred-millisecond authorisation. An analyst team that can review about two thousand cases a day. And I want the cost of a wrongly declined customer, because that number sets my threshold and I suspect nobody has computed it.

Next: the third option that turns the threshold into a dial.