Near-Optimal Dimension Lower Bounds for Single-Vector Embeddings of Maximum Inner Product Similarity

2026-07-22

AI Summary

This work investigates the lower bound on the embedding dimension required for single-vector embeddings to approximate maximum inner product similarity (MAX-IP) within additive error ε. By constructing specific query and document point sets on the unit sphere with a separation margin of Ω(ε), leveraging Sherstov’s pattern matrix, constant-width DNF formulas, block encoding, and normalization techniques, the authors establish that for any δ ∈ (0,1) and sufficiently large dataset size m, the embedding dimension D must satisfy D ≥ m^{c_δ/ε^{2−2δ}}. This result provides the first strong lower bound exhibiting an almost optimal 1/ε² dependence, nearly closing the exponential gap between prior upper and lower bounds, and applies even to data-dependent embedding schemes.

Abstract

Multi-vector embeddings represent items by point clouds and compare query and document point clouds using Chamfer similarity, whereas single-vector embeddings use ordinary inner products. For singleton queries, Chamfer becomes maximum inner product similarity (MAX-IP). In our setting, MUVERA gives dimension $m^{O(1/\varepsilon^2)}$~\cite{dhulipala2024muvera}, whereas the previous lower bound $(\varepsilon^2m)^{Ω(1/\varepsilon)}$~\cite{jayaram2026expressive} left a gap between $1/\varepsilon$ and $1/\varepsilon^2$ in the exponent of $m$. We nearly close this gap. For every fixed $δ\in(0,1)$, there are constants $A_δ,c_δ>0$ such that, for all sufficiently small $\varepsilon>0$ and every $m\ge(1/\varepsilon)^{A_δ}$, there exist unit query vectors and document point clouds of at most $m$ unit vectors for which every single-vector approximation of all pairwise MAX-IP values to additive error $\varepsilon$ has dimension \[ D \ge m^{c_δ/\varepsilon^{2-2δ}}. \] This holds even for fully data-dependent representations chosen after seeing the dataset. It also applies to Chamfer because all queries are singletons. Since $δ$ can be arbitrarily small, the exponent approaches the $O(1/\varepsilon^2)$ dependence of the upper bound. The proof combines Sherstov's pattern matrix method with polynomial-size, constant-width DNF formulas computing functions of approximate degree $Ω(k^{1-δ})$. Uniform-width padding and a block encoding create an $Ω(\varepsilon)$ gap. A dummy coordinate then equalizes all false inputs, yielding a unit-sphere MAX-IP matrix that is an exact two-valued affine image of the DNF pattern matrix with gap at least $8\varepsilon$. This allows the approximate-rank bound to apply.

Problem

• maximum inner product similarity
• dimension lower bounds
• single-vector embeddings
• approximate rank
• Chamfer similarity

Innovation

• maximum inner product similarity
• dimension lower bound
• single-vector embedding
• approximate rank
• DNF formula