Meta aggregates stranded idle GPU capacity into an opportunistic pool for ads ranking
Meta_Engineers · x · 2026-10-29
Meta's engineering team details how it tackles the inference capacity paradox: fleets can be globally constrained while meaningful pockets of capacity sit locally idle, stranded by peak provisioning, regional resource ratios, and accelerator-type mismatches. Those pockets are aggregated into a shared opportunistic capacity pool and spent on ads recommendations via two elasticity levers:
- Rank More Ads (RMA): increases candidates evaluated by ranking stages, requiring both CPU and accelerator surplus.
- Serve Better Models (SBM): routes a larger share of requests to more powerful models, consuming accelerator-only surplus RMA can't use.
Because the pool is preemptible, Meta keeps baseline serving within latency and reliability targets through physical and logical isolation, PID-controlled feedback loops, and millisecond value-aware load shedding at both server and client. A full engineering blog is linked — a solid first-party deep dive into inference fleet capacity scheduling.
More from Infra
- Short seller claims Cerebras wafers degrade after ~2 years, undermining its business — himanshustwts · 2026-10-29
- Apple's RAM upgrade pricing: 512GB costs a staggering $13,950 over base — firstadopter · 2026-10-29
- AI data center gas buildout could cost over $5.6 billion a year in health impacts — AINowInstitute · 2026-10-29
- Report: 177GW of planned gas plants to power US AI data centers, ~80% gas-fired — AINowInstitute · 2026-10-29
- Analysts split on Google's merchant TPU: $100B vs $252.7B by 2028 forecasts — Beth_Kindig · 2026-10-29
- Gemini 4 Carbon costs $7.76 per task, 23% less than Claude Opus 5.5 at same intelligence — ArtificialAnlys · 2026-10-29