Meta's ROCS Paradigm Boosts Recommendation Retrieval QPS Up to 3x
_reachsumit · x · 2026-07-31
Meta released a paper titled "ROCS: Request-Oriented Compute Sharing for Efficient Large-Scale Recommendation".
- Core Problem: Modern recommendation models improve prediction quality by scaling feature-interaction and sequence modules, but high production costs cap system scalability.
- ROCS Paradigm: Exploits the property that each user request evaluates many candidates. It defers request-candidate interactions as late as possible, isolates candidate-dependent representations, and evaluates substantial portions of the model once per request rather than once per candidate.
- Implementation: Develops Generalized Layer Masking (GLM) to enforce candidate isolation in feature-interaction architectures.
- Results: Significantly reduces redundant compute, boosting retrieval QPS (queries per second) by up to 3x while maintaining or improving prediction quality.
More from Infra
- Open-source Rust GGUF runtime runNburn runs 295B model on 64GB RAM, 2.8x faster decode than llama.cpp — coderyeon · 2026-07-31
- Amazon Says AI Demand Is in Early Stages, Projects Over $220B CapEx for 2026 — inductionheads · 2026-07-31
- Stanford Paper Proposes Mandatory Resource Disclosure for Hyperscale Data Centers — ruthstarkman · 2026-07-31
- Serving 2.8T Param Models Gets Cheap: $1.75/M Tokens on AMD MI355X — scaling01 · 2026-07-31
- Open Source Ternary LLM Engine Tritium: Slashes VRAM Usage and Outperforms llama.cpp — Wide_Big_6969 · 2026-07-31
- Inside MiniMax: RL Lead Details Infrastructure and Training for Agents at Scale — AI Engineer · 2026-07-31