Maximum inner product is not nearest neighbour

Some embedding models want inner product rather than cosine, and most vector databases will happily give you an inner-product index. It looks like the same problem with a different formula. It is not the same problem, the difference is structural rather than cosmetic, and it is the clearest example on this site of a geometric property being load-bearing for an algorithm.

The naive assumption

You have a similarity function. Bigger means more similar. Search means “find the largest”. Surely any index that finds the nearest points under Euclidean distance can find the largest points under inner product — it’s the same traversal, you just flip the comparison.

maximise  a · b = Σᵢ aᵢ bᵢ        instead of        minimise  d(a, b) = ‖a − b‖

How it fails: every index in this field prunes, and every pruning argument is a triangle-inequality argument. Inner product doesn’t satisfy the triangle inequality. It doesn’t satisfy the other axioms either, and the failures are not technicalities.

Exactly which properties break

Recall what a metric requires: non-negativity, identity, symmetry, and the triangle inequality. Inner product as a similarity fails two of the four in ways that matter.

A point need not be its own best match. Under a metric, d(a, a) = 0 and nothing is closer to a than a. Under inner product, a · a = ‖a‖², so a different vector b pointing roughly the same way but much longer will score higher: a · b > a · a. The consequence is immediate and strange — self-similarity is not maximal. The maximum-inner-product neighbour of a point is often not that point.

There is no triangle inequality, so no bound propagates. This is the fatal one. With a metric, if you know d(a, b) and you have measured d(query, a), you get a free lower bound on d(query, b) and can discard b unexamined. Inner product gives you nothing comparable. Knowing that a and b are similar to each other, and that the query scores poorly against a, tells you very little about how the query scores against b, because b might simply be longer.

Long vectors are universal answers. The highest-norm vectors in the collection score well against almost every query, since the norm multiplies into every score. A handful of vectors becomes the top result for large parts of query space. This is a severe form of hubness, and it is a property of the objective, not of the data.

What that does to each index family

Graph indexes. A proximity graph is built by connecting each point to its nearest neighbours under the search metric, and the search does greedy descent, moving to whichever neighbour improves the score. Under inner product two things go wrong. The graph is now a graph of “who has high inner product with whom”, which — because self-similarity isn’t maximal — is a much stranger relation than proximity; it is not even reflexive in the useful sense. And greedy descent’s guarantee that local improvement leads somewhere relies on the score surface having the geometry of distance to a point. Under inner product the surface is a linear function, which is maximised at the boundary of the data, so the search wants to run to the extremes rather than converge to a location. Implementations that support inner product directly do so with a modified construction, not by relabelling the comparison.

Cluster indexes. Assigning vectors to the nearest centroid and probing the centroids closest to the query is a distance argument through and through. Under inner product, a cluster whose centroid scores badly can still contain the global maximum, because a member with a large norm can beat the centroid’s own score by a wide margin. Ranking clusters by centroid inner product is therefore a much weaker heuristic than ranking them by centroid distance. Practical implementations rank by a bound that accounts for the maximum norm within each cluster instead — which works, and needs that extra statistic stored per cluster.

Quantization. Product quantization approximates a vector by a concatenation of codebook entries, and the approximate score is a sum of table lookups. This part survives fine — inner product decomposes across sub-vectors just as squared Euclidean distance does. What suffers is error behaviour: an inner-product error scales with the query’s magnitude in that subspace, so the error is not uniform across queries the way a distance error roughly is.

The reduction that fixes it

Here is the trick that lets a metric index solve the inner-product problem, and it’s worth seeing because it explains why the two problems are related at all.

Write out the squared distance:

‖q − v‖² = ‖q‖² + ‖v‖² − 2 (q · v)

For a fixed query, ‖q‖² is a constant, the same for every candidate. So minimising distance is the same as maximising (q · v) − ‖v‖²/2. That is inner product penalised by the candidate’s norm — which is precisely the difference between the two problems. Nearest neighbour prefers vectors aligned with the query and short; maximum inner product prefers aligned and long.

That also shows how to convert one into the other. Append a dimension carrying the norm. Let M be the largest norm in the collection, and transform:

v ↦ ( v ,  √(M² − ‖v‖²) )         every stored vector, now with one extra component
q ↦ ( q ,  0 )                    every query, padded with a zero

Now every transformed stored vector has norm exactly M — the padding tops up whatever the original was short by — and the extra component contributes nothing to any inner product with a query, since the query’s entry there is zero. Inner products are unchanged, and all the norms are equal. When all norms are equal, distance and inner product rank identically, so a standard metric index over the transformed vectors returns exactly the maximum-inner-product answers.

This works, it is exact, and it is mostly of theoretical interest, because it has an unpleasant practical property: the transformed data is now concentrated on a sphere of radius M with a wide spread of values in the padded dimension, which tends to make the geometry harder for the index than the original data was. A few high-norm outliers set M and squash everything else. It’s the right thing to know and rarely the right thing to deploy.

The shortcut almost everyone should take

Normalise the vectors and the whole problem disappears.

If every vector has unit norm, then ‖v‖² is a constant too, the penalty term in the identity above is the same for every candidate, and inner product, cosine and Euclidean distance all rank identically. Maximum inner product is nearest neighbour. Every index property that depends on metric structure is restored, and the engine can compute the cheap dot product while giving you cosine semantics.

So the decision reduces to one question: does your embedding model put meaning in the magnitude?

Most general-purpose text embedding models don’t — many emit unit vectors already — and for those, normalising is free and you should stop thinking about this. Some models are trained so that norm carries a notion of importance, term weight or confidence, and learned sparse retrieval models deliberately use magnitude as weight. For those, normalising discards signal the model was built to convey, and genuine maximum inner product search is what you need.

If you do need it, the things to check are specific and mechanical:

• Does the engine implement inner product natively, or by normalising behind your back? These are different, and if it normalises, you didn’t get what you asked for.
• How does its cluster ranking or graph construction account for norms? A native implementation has to do something here; if the documentation doesn’t say, the correctness of the pruning is unknown.
• Look at the norm distribution. A few extreme norms dominating the collection is the condition under which inner-product search is most degenerate — a handful of vectors answering everything — and it is measurable before you index anything.

The general lesson is the one worth carrying: an index is not a container for an arbitrary scoring function. It is a set of algorithms that exploit specific geometric properties, and recall is only meaningful against a ground truth computed under the same objective the index thinks it is optimising.