RL post-training compute accounting: one weight update per episode blurs the train/inference line
davidmanheim · x · 2026-10-13
Responding to Ben Todd, David Manheim notes that much post-training RL compute isn't really inference-time compute: the weight update is the larger computational expense, but since it happens only once per RL episode, amortized per episode it may reasonably count as inference compute. A nuanced take on how to classify RL compute in training-vs-inference estimates.
More from Infra
- Parsing JSON at tens of GB/s in Python: free-threaded 3.14 hits 65.7 GB/s on 64 cores — charliermarsh · 2026-10-13
- IBM moves Spyre accelerator to PyTorch Inductor, cutting compiler code 7x — PyTorch · 2026-10-13
- Your servers can report high availability while clients can't reach them — Franc0Fernand0 · 2026-10-13
- Model weights are only 1 of 4 VRAM allocations: why loaded LLMs still OOM — blaizedsouza · 2026-10-13
- US electricity chart: 4,079 TWh in 2025, data-center servers still a small slice of the pie — skorusARK · 2026-10-13
- While the US fights over data centers, China is building them fast — GeneReddit123 · 2026-10-13