Similarity Metrics and Normalisation
In one line: three metrics are in common use, they become equivalent once vectors are normalised, and the only real mistake is using a different one than the model was trained with.
The three
Cosine similarity measures the angle between two vectors and ignores their length. Range −1 to 1, where 1 is identical direction. It asks are these pointing the same way.
Dot product multiplies the vectors element-wise and sums. It is affected by both angle and magnitude, so a longer vector scores higher against everything.
Euclidean distance is straight-line distance in the space. Smaller is more similar, which inverts the comparison direction and is a small but real source of bugs.
Normalisation collapses the distinction
Scale every vector to length 1 and magnitude stops carrying information. Cosine and dot product become identical, and Euclidean distance becomes a monotonic function of cosine — so all three produce the same ordering.
That is why the choice matters much less than people expect, and why most embedding models normalise their outputs by default. Once normalised, this is not a decision.
The practical rule: check whether the model normalises. If it does, use dot product, because it is the cheapest to compute and mathematically identical to cosine at unit length.
When magnitude actually carries meaning
The interesting case is when you deliberately do not normalise.
In some models — particularly those trained for retrieval with certain objectives — vector magnitude correlates with something real, such as how much information the passage contains or how confident the encoding is. Normalising throws that away.
| Normalise | Do not normalise | |
|---|---|---|
| Magnitude | Discarded | Contributes to the score |
| Long documents | Compete on equal terms | May score higher simply for being long |
| Metric choice | Irrelevant — all three agree | Cosine and dot product diverge |
| Use when | The default, and what most models expect | The model documents that magnitude is meaningful |
The second row is the one that bites. With unnormalised vectors and dot product, longer passages tend to have larger magnitudes and therefore win, which shows up as a retrieval system that inexplicably prefers long documents. It looks like a relevance problem and is a normalisation problem.
The index has to agree
A subtler version of the same mistake. Vector stores are configured with a distance metric at index creation, and that configuration is what the graph or partition structure is built around.
If you build an index with Euclidean distance and query it expecting cosine, results are wrong in a way that is hard to spot — the index is internally consistent, it just organised the space around a different notion of nearness.
Three places to agree, and the failure of any pairing is silent. It is worth a single line in a design review.
A note on high dimensions
Distances behave counter-intuitively in high-dimensional spaces: as dimensionality grows, the distance between the nearest and farthest points in a random set converges, and every point starts to look roughly equidistant from every other.
This is often raised as an objection to vector search, and the reason it does not sink the technique in practice is worth knowing. Embeddings are not random points — they occupy a much lower-dimensional structure inside the nominal space, because the training objective deliberately clusters related items. The effective dimensionality is far below the stated one.
The practical residue is real though: absolute similarity scores are not interpretable across queries. A cosine of 0.82 means something different for one query than another, which is why a global relevance threshold usually behaves badly and why relative ranking is the reliable signal.
Key takeaway
Cosine, dot product and Euclidean distance produce identical rankings once vectors are normalised, so the choice is usually not a decision — use dot product on normalised vectors because it is cheapest. The real requirement is agreement between the model's training objective, the index's configured metric and the query-time comparison, because a mismatch degrades retrieval a few points with no error anywhere. And absolute similarity scores are not comparable across queries, so set any relevance threshold empirically and revisit it whenever the model changes.
Next: why you cannot simply compare against everything.