Apple's ICML paper: learned support functions speed up RAG retrieval up to 8x

大模型之路 · wechat · 2026-11-19

An Apple ML paper accepted at ICML 2026 replaces centroid-based IVF routing with learned support functions. Exploiting convex analysis — a cluster's best achievable score is a convex function of the query whose gradient points to the optimal key — it trains two small networks offline: SupportNet (a router scoring each cluster's upper bound) and KeyNet (a query rewriter feeding existing indexes like FAISS untouched). On Natural Questions (2.6M passages), SupportNet hits 95% recall 1.9x faster; KeyNet reaches 99% recall up to 8x faster scanning 11x fewer keys.

Caveats: the largest network is 20% of database size, results are vendor-reported on one dataset, and the method only works when queries follow the training distribution — requiring real query logs and stable query patterns. HN commenters half-argue that BM25/grep is often the better answer for LLM agents. Code: apple/ml-amips (arXiv 2603.08001).

Original post →

More from Infra

Infra channel →