Coordinated Behaviour

In one line: a thousand accounts each posting something unremarkable, in concert, is a violation that no individual post commits.

The category that breaks per-post moderation

Most of this chapter assumes a post can be judged. For a large class of real harms that is false.

Coordinated inauthentic behaviour — a network of accounts presenting as independent people while acting as one. Each post is ordinary political speech, product praise, or commentary. The violation is the deception about who is speaking.

Manufactured amplification. Content promoted to appear organically popular. Every engagement is a real account performing a real action.

Brigading. Coordinated targeting of a person or a piece of content, where each individual message might be within policy and the aggregate is harassment.

Spam operations. Individually mundane posts distributed across many accounts to evade per-account rate limits.

What to look for

The signal is structure and timing rather than content, and it is the same family of techniques as fraud-ring detection applied to a different behaviour.

Shared infrastructure. Devices, IP ranges, registration patterns, phone numbers, recovery emails. Running genuinely independent infrastructure per account is expensive, so operations reuse.

Synchrony. Accounts created in the same window, posting within seconds of each other, engaging with the same content in the same order. Real independent users do not act in lockstep.

Content similarity. Identical or near-identical text across accounts, the same media with small variations, or the same message paraphrased — which is what embedding similarity catches and exact matching does not.

Behavioural templates. Accounts following the same lifecycle — a dormant period, a burst of innocuous activity to build history, then the campaign.

Interaction structure. Networks that engage overwhelmingly with each other rather than with the wider platform. Real accounts have messy, outward-facing interaction graphs.

The strongest signals are combinations. Similar content alone is a coincidence or a trend; similar content from accounts created the same week on the same devices, posting in synchrony, is an operation.

The detection shape

Clustering rather than classification, and that changes the pipeline.

Build a graph over accounts with edges for shared infrastructure, temporal correlation, content similarity and interaction. Find dense components. Score each component on how implausible its cohesion is for an organic group.

Run offline, on a schedule. This is a batch job over the population, not a per-request decision — and the output is a network label attached to accounts, which the online path can then read as a feature.

Enforcing against a network

Different from enforcing against a post, and the differences matter.

Act on the whole cluster at once. Removing accounts one at a time tells the operator which signals you detected, and they adapt. Simultaneous action across the network is both more effective and less instructive.

Timing is a decision. Acting immediately stops the harm and reveals the detection. Observing longer maps the operation better and lets it continue. There is no universal answer, and the trade should be explicit rather than defaulted.

The bar is higher. Removing a network is a large action affecting many accounts, some of which may be genuine people who joined something they did not understand. Network enforcement should require higher confidence than post enforcement, and usually human confirmation.

That caveat is the one to raise unprompted, because the naive version of this feature is genuinely dangerous.

The adversarial dynamic

Same as fraud, and the same lesson applies.

Operators observe what gets caught and adapt: rotating infrastructure, adding jitter to posting times, paraphrasing rather than copying, ageing accounts for months before use.

So the durable signals are the ones that are expensive to fake, and the useful distinction is between what an operation can vary cheaply and what it cannot. Posting times are free to randomise. Building a genuine, messy, outward-facing interaction graph over months is not — which is why interaction structure survives longer than timing correlation.

And the same monitoring consequence: a sudden drop in detected networks is more likely a detection failure than a genuine decline.

When this is over-engineering

Worth naming, because it is a substantial investment.

If the platform's problems are individual bad actors, this machinery catches nothing that per-post moderation misses. Building an account graph over hundreds of millions of nodes to catch a handful of spammers is effort better spent elsewhere.

The signal that it is warranted is evidence of organised activity — repeated campaigns, spam that survives per-account rate limits, engagement patterns that do not look organic. Until that exists, the per-post system plus rate limiting covers the ground.

The connection back

Two threads worth pulling together.

This is the fraud chapter's graph work, in a different domain. Shared infrastructure, dense components, synchrony, degree anomalies. The techniques transfer directly, which is worth saying out loud — it shows the pattern is general rather than memorised per-problem.

It is the reach argument again. A coordinated network exists to manufacture reach, so its harm scales with the amplification it achieves. Which means the priority for investigating a suspected network is the same function as everywhere else: how much exposure is it producing?

Next: telling people what you did, and letting them argue.