What cosine similarity between embeddings actually means

Cosine similarity measures the orientation of two vectors once their lengths are ignored. It is a geometric relationship inside the particular embedding space that produced them, and not a universal unit of meaning, relevance, or usefulness.

For a query vector q and a candidate vector d:

cos(q, d) = (q · d) / (||q|| ||d||)

The numerator is the dot product; the denominator removes the effect of magnitude. A small angle produces a higher score, perpendicular vectors score zero, opposite directions score negative. Those facts are exact. The interpretation of the angle is learned, and nothing in the formula guarantees it.

Is cosine similarity a measure of meaning?

It is a measure of position. An embedding model maps an input to coordinates, cosine compares those coordinates, and the training objective decides which relationships end up geometrically close.

Steck, Ekanadham and Kallus show in Is Cosine-Similarity of Embeddings Really About Similarity? (WWW 2024) that cosine similarity can yield arbitrary, and therefore meaningless, similarities. Their result is analytic rather than a benchmark: there is no accuracy delta attached to it, and it does not show that every embedding model fails.

That boundary is worth stating plainly, because the paper is often over-read. Cosine can order candidates perfectly well when model, data, metric and task happen to align. The claim is that you cannot infer that alignment from a score. A precise-looking value still needs an interpretation supplied by training and validated against your application.

Why does the paper say the similarities can be arbitrary?

The argument is about underdetermination and regularization. For some regularized linear models the resulting similarities are not even unique. For others, the choice of regularization implicitly controls them. The geometry is therefore not pinned down by the task behavior you can observe.

Deep models stack several forms of regularization, including effects nobody chose as a definition of semantic similarity, and the authors argue these combine to render cosine results opaque and possibly arbitrary.

"Arbitrary" here does not mean random in every implementation. It means similarity values can depend on modeling choices while carrying none of the stable semantic interpretation engineers routinely assign them. A high score is evidence that two vectors are close under the current geometry. By itself it is not evidence that two records satisfy your definition of relevance.

Why can the same score mean different things?

Because a score is meaningful only relative to its own space. The same number from two models does not imply the same relationship between the underlying texts, images, users or products. Model identity is part of the score.

A model trained for topical retrieval puts documents about one subject near each other. A model trained for classification, recommendation or stylistic matching organizes the same inputs differently. "Similar" means close under the objective that shaped the vectors, which is not automatically the relevance your product needs.

The same applies across versions. A new checkpoint can reorganize the space while preserving good task behavior, so old and new cosine values describe different geometries and comparing their raw thresholds treats unlike measurements as interchangeable.

Are cosine thresholds portable?

Not by default. "Keep every result above 0.8" is a statement about one embedding space, one vector-generation configuration and one candidate population, not a general relevance boundary.

Thresholds also encode the candidate distribution. A fixed cutoff can retain too few results in one corpus and too many in another with the model completely unchanged, because the threshold is quietly describing corpus properties alongside model geometry.

So treat a threshold as a calibrated decision rule with provenance. Record the model version, preprocessing, normalization behavior, corpus, query distribution and the labels used to pick it. Recalibrate when the model or version changes, and recheck after any major corpus change.

Should semantic search rank or threshold?

Rank first. Ranking asks which candidates are closer within the current space, which is a question the metric can answer. Thresholding asks whether a score crosses an absolute boundary, which needs evidence that the boundary tracks your notion of relevance.

A ranked list still does not prove relevance; it orders candidates by the chosen metric and nothing more. Real evaluation means labeled queries, known relevant documents, and the operating conditions of the system. Eyeballing a handful of scores is not an evaluation protocol.

When a binary action is genuinely required, calibrate the cutoff on representative labeled pairs and choose it against the costs of false positives and false negatives, rather than because the decimal looks reassuring. Then repeat after changes to the model, index, preprocessing or corpus.

When should you add a reranker?

When first-stage vector retrieval finds a good candidate set but cosine does not express the final judgment. The first stage optimizes recall and speed; a task-supervised or cross-input model then inspects query and candidate together, using a signal closer to the decision your product makes.

This is not an admission that cosine is defective. It separates two jobs: finding plausible candidates in a vector space, and deciding which of them satisfy a specific request. The second stage deserves its own scrutiny, because a reranker can promote exactly the lookalike passages that hurt most. The second often needs exact constraints, authority, recency or permissions, none of which a generic embedding encodes.

How retrieval fits among the other ways an agent reaches data is a system-design question covered in MCP vs API vs RAG. That comparison does not give a similarity score a universal meaning either.

What should you record with a score?

Enough context to reproduce its interpretation: the embedding model and version, input preparation, similarity metric, normalization behavior, corpus snapshot, and the decision rule applied.

Without that, a score log is hard to compare and easy to misread. A 0.8 in one experiment may be a ranking signal while the same value elsewhere is a thresholded production decision. The decimal does not identify which claim is being made.

Input construction matters too, since it determines what the vector represents. Retrieval quality degrades when a system supplies more context than a model uses well, which is a separate failure mode worth keeping distinct so that a context problem does not get recorded as evidence about cosine.

The defensible reading of a score is local: these two inputs are close under the current model, vector-generation process and metric. Anything stronger needs labeled evidence from the task consuming the result. Cosine similarity remains genuinely useful as geometry. The mistake is promoting geometry into a universal scale of meaning.