When Does Dense Retrieval Need Asymmetric Geometry? A Bias-Variance Theory of Shared and Dual Projections
Maojun Sun, Yancheng Yuan, Jian Huang, Ruijian Han
cs.IR, cs.LG
2026-09-26
Dual is better iff directional signal covers extra DoF. Shared wins 13/16 cells at n=32; Dual sweeps n≥1024. CARS hits 90.1% accuracy, cutting regret 49-96%.
Dense retrieval scores a query against documents with inner products of vectors. When you adapt frozen embeddings, you can share one rank-r projection P on both sides, which forces the scoring operator M=P⊤P to be positive semidefinite, or fit two projections A and B so that M=A⊤B can be any matrix of rank at most r.
The extra freedom cuts approximation bias and also opens extra directions for training noise. Metric-learning and dual-encoder papers have studied both families. What they did not give is a finite-sample rule for when directional signal is strong enough to pay for Dual's extra estimation error. This paper writes that rule as a bias-variance boundary for low-rank bilinear scoring.
The score gap on a (query, positive, negative) triple is the Frobenius inner product ⟨M, XD⊤⟩. Shared lives in the symmetric PSD matrices of rank at most r; Dual lives in all rank-at-most-r matrices. Theorem 1 gives the exact Shared approximation error as three terms: skew energy, negative eigenvalues, and dropped positive eigenvalues. Subtract Dual's singular-value tail and you have the exact approximation gap.
With finite samples Dual also fits noise in those extra directions. They run a local Gaussian experiment around a rank-r Shared operator: the observation is the target plus isotropic noise of scale σ/√n, and the directional departure is also order 1/√n, so bias gain and variance cost sit on the same scale. The tangent spaces differ by k=r(2p−r−1)/2 Dual-only directions, split into skew motion inside the active r-plane and unmatched left/right cross-subspace motion. Corollary 1: Dual has lower local risk iff the signal energy δ² on those directions exceeds σ²k. Extra degrees of freedom need a repeatable directional gap; otherwise Shared is the stabler fit.
Raw fitted asymmetry is biased upward by noise. In the Gaussian model, SURE corrects it and selects Dual iff the observed Dual-only energy exceeds 2σ²k. With no directional signal, the chance of picking Dual is at most (2/e)^{k/2}. Real embeddings do not come with a known geometry or noise variance, so CARS splits training pairs in half, keeps Dual-versus-Shared structure that repeats across halves, and charges the halves' disagreement as sampling variance. The expected score equals (δ²−σ²k)/n, the same comparison as the local boundary.
Encoders: E5-base-v2, BGE-base, GTE-base, Contriever-MSMARCO. Retrieval metric: test NDCG@10 with exact full-corpus inner-product search.
The synthetic two-view grid shows the predicted crossing. At rank 8, a 0.8-radian rotation first turns Dual-positive at n=256; 1.2 radians already turns at n=64. A local-Gaussian calibration matches Theorem 2's risk limits within 0.25% and Theorem 3's SURE selection probabilities within 0.001.
On frozen real embeddings they rotate only queries, leaving documents and labels fixed. Averaged over five datasets, the Dual-minus-Shared NDCG@10 gap rises from about 0.0032 at 0° to about 0.0146 at 90°, a 4.6-fold increase. On MS MARCO the gap flips from about −0.0008 to +0.0219. The abstract says the mean advantage more than doubles; the body number is 4.6 times.
| Setting | Comparison | Result |
| n=32, 16 cells | Shared vs Dual | Shared wins 13/16 |
| n=1024 and 2048 | Shared vs Dual | Dual wins all 32 cells |
| Query rotation 0° to 90° | Dual-Shared NDCG@10 | 0.0032 to 0.0146 |
| 168 operator-risk curves | more training queries | all move toward Dual |
| CARS vs fixed geometry | held-out regret | down 49-96% |
| CARS geometry choice | accuracy / outer-fold wins | 90.1% / 85/100 |
Raising rank from 4 to 32 barely moves absolute NDCG@10: Shared 0.7242 to 0.7257, Dual 0.7282 to 0.7289, with no extra Dual advantage. Capacity is not the lever. Sample size is.
CARS chooses geometry from training data alone. Against the better of the two fixed rules, held-out regret drops on all four encoders; it wins 85 of 100 encoder-fold comparisons. Always Dual hits regret 9.51 on MS MARCO with BGE-base; Always Shared hits 0.496 on FEVER with Contriever. Locking one geometry bills you on the datasets where it is wrong.
The contribution is a switch for projection geometry, not a leaderboard retriever. Use Dual when directional mismatch is strong and you have enough pairs; default to Shared at small n. Anyone attaching a low-rank adapter on frozen BGE or E5, or training a RAG retrieval head, can pick geometry from training size, or run CARS before burning a validation split.
Rank 4 to 32 buys almost no NDCG. More labeled queries flip the winner. Budget belongs on samples, not on a wider adapter.
The boundary assumes bilinear inner-product scoring and local isotropic Gaussian operator noise. Cosine scoring or strongly anisotropic noise means the formula is not a drop-in rule.
The authors flag three. The exact risk boundary assumes local, isotropic Gaussian operator noise. They have not extended it to anisotropic noise, or to ranking metrics such as NDCG. Partial sharing, with only some directions left unshared, is left as future work.
CARS's 90.1% accuracy is computed on held-out operator Frobenius loss, not on full-corpus NDCG. Table 1 and the retrieval figures are different protocols. The paper does not show that the geometry CARS picks is the one with higher NDCG.
Query rotation is an intervention: queries are spun through relevance-informed planes up to 90° to amplify cross-view mismatch. Whether natural domain shift traces the same curve is untested. Absolute NDCG gaps mostly sit between 0.003 and 0.015. They are measurable on the full corpus; whether they are large enough to feel in production is not discussed.
Appendix H adds an identifiability catch. The same bilinear operator M=A⊤B can be realized with very different projected norms, and cosine ranking can reverse. Operator-level claims are tied to bilinear scoring. If production uses cosine, the theory does not transfer as written.