Google's Retrieve-for-Train cuts complex AI search from ~50s to sub-second
aigclink · x · 2026-09-16
Google Research unveiled Retrieve-for-Train, an ICML 2026 framework that moves AI search's "thinking" from inference time to offline training, cutting large-scale query latency from 50 seconds to sub-second—12-20x faster than autoregressive approaches. The method uses offline RL to compile expert retrieval behavior into a 53.9M-parameter diffusion model that generates an entire slate of retrieval targets in embedding space with a single parallel pass, bypassing expensive text CoT generation. It improves set-level quality (returning complementary slates like tent+sleeping bag+stove instead of ten near-identical tents) while removing the inference bottleneck that makes complex AI search impractical for real-time search boxes.
More from Infra
- Big Tech's Future Data Center Leases Top $1.3T, 70% Added in Last Four Quarters — rohanpaul_ai · 2026-11-02
- Daniel Lemire: AI is exploding demand for boring compute, server CPUs to double — lemire · 2026-11-02
- Physical AI's real story: a 4-layer map of 15 companies behind humanoid robots — pdamodaran · 2026-11-02
- Under 5% of US consumers pay for AI, yet compute demand is already forcing usage cuts — NinaDSchick · 2026-11-02
- Sam Altman: AI query energy cost fell 1,000x-100,000x in 3-4 years — rohanpaul_ai · 2026-11-02
- Marvell and Lumentum to merge, targeting $550B AI connectivity market — BenBajarin · 2026-11-02