Rollout scheduling: the underappreciated infra trick boosting inference efficiency
stochasticchasm · x · 2026-09-22
Commenters highlight a favorite trick from a recent AI infra paper: rollout scheduling, which can have an outsized effect on inference efficiency. The paper's decision to publish its metrics publicly also drew praise as unusually transparent.
More from Infra
- 456GB DeepSeek v4.1 runs locally at 40 tok/s with Threadripper + dual RTX 6000 hybrid setup — HankYeomans · 2026-09-22
- Agent Substrate roadmap: sub-second suspend/resume runtime for dense agent deployments — rakyll · 2026-09-22
- StepFun's Step 5 Preview scores 44 on AA Intelligence Index at ~2.8x lower cost than Kimi K3 — ArtificialAnlys · 2026-09-22
- SemiAnalysis tears down A20 on TSMC N2, DRAM revealed in iPhone 18 Pro Max package — dylan522p · 2026-09-22
- Bernstein's Intel road notes: servers sold out through 2027, real test comes in 2028 — BenBajarin · 2026-09-22
- Grok reportedly hit by a datacenter incident, details still unclear — Daniel_Farinax · 2026-09-22