Storage Layout and Tiering

In one line: the object key is part of the design, and the cheapest storage class is usually the wrong one here.

The key is a backup of the index

Six hundred million objects need names. The naming scheme is worth a minute of thought because it determines whether the archive survives losing its database.

A key built from identity and capture time is self-describing:

site-12/cam-0417/2026/09/03/14/20260903T142204120Z-30860ms.fmp4
└─site─┘ └cam──┘ └──date──┘ hr └── start instant ──┘ └duration┘

Every field a query needs is recoverable by listing the bucket. If the index is lost, a scan reconstructs it — slowly, but completely. An opaque UUID key gives you objects that exist and cannot be identified, which is the same as having lost them.

The hierarchy also gives cheap operations for free. Deleting a day for one camera is a prefix operation. Auditing whether a camera recorded on a given afternoon is a prefix listing. Restricting an operator to one site is a prefix-scoped policy.

Time is not one thing

Two timestamps matter and designs that conflate them break on backfill.

Capture time is when the camera recorded the footage. Arrival time is when the gateway received it. Normally they differ by a second. After an uplink outage, footage captured on Tuesday arrives on Thursday, and the two differ by days.

Everything user-facing must key on capture time — the object path, the index, the retention clock. Arrival time is operational metadata, useful for spotting backfill and for debugging, and it must never drive placement or expiry. A retention job that deletes by arrival time will keep backfilled footage days too long, and one that places objects by arrival time makes Tuesday's footage unfindable under Tuesday.

This also means camera clocks matter. A camera with a drifting clock writes footage under the wrong time, and the operator searching 3pm finds nothing. Cameras should sync via NTP, the gateway should record the observed skew, and large skew is a health alert rather than something to silently correct.

Tiering, and the minimum-duration trap

The instinct is to move older footage to progressively cheaper classes. For a thirty-day window that instinct is mostly wrong, and knowing why is a strong signal.

Cheaper classes carry minimum billable durations. Infrequent-access tiers bill a 30-day minimum per object, archive-instant tiers 90 days, and deep archive 180. Delete before the minimum and you are billed for the remainder anyway.

Against a 30-day retention policy the arithmetic collapses:

There is a second cost that bites here. Transitioning an object between classes costs a request per object, and with six hundred million objects a fleet-wide transition is a large bill for a small saving. Guidance discourages transitioning objects below 128 KB for exactly this reason, and short video segments can land near that boundary.

So the rolling window lives in Standard, possibly Standard-IA, and that is the honest answer.

Two populations, two policies

Tiering does pay — just not on the rolling window. It pays on the footage that is kept because someone decided it mattered.

The flagged population is tiny — a fraction of a percent of footage — and it is the only part with a lifespan long enough to amortise a minimum duration. It is also the part with real value, so it justifies a different durability posture: cross-region replication for the incidents, single-region for the rolling window nobody will ever read.

This split is the answer to the earlier retention question. "Thirty days of everything" and "years of what matters" are different systems sharing an ingest path, and separating them is what makes the cost defensible.

Compaction, and why it is tempting

Six hundred million small objects is a lot of metadata. An obvious optimisation is to compact yesterday's segments into hour-long objects — 24 per camera per day instead of 21,600 — cutting object count by three orders of magnitude and making a transition to a cheaper class affordable.

The costs are real. Compaction is a full read and rewrite of every byte, so it doubles the write volume and incurs its own request charges. It also coarsens deletion: an hour-long object cannot be partially expired, so per-segment retention precision is lost. And random seek within an hour object requires byte-range reads plus an index of where each segment starts inside it.

It is worth doing when the retention window is long and object count has become the binding constraint. It is not worth doing on a 30-day rolling window where the objects expire before the savings accrue. Naming the trade rather than reflexively proposing compaction is the better answer.

Next: deleting on schedule, which is harder than it sounds.