Teacher-trainer RL training keeps Monte Carlo KL speed via fused blockwise reverse kernel
brandondamos · x · 2026-11-08
brandondamos explains why his teacher-trainer setup retains the same speed as the Monte Carlo KL estimate: transporting hidden states between teachers and trainers adds only minor staleness to traces without hurting generation or teacher-annotation throughput, and the fused blockwise reverse kernel on the trainer side is memory- and compute-efficient.
More from Research
- Debate over METR's 2x speedup claim: does it lean on private information? — JacquesThibs · 2026-11-08
- Scaling May Be Superlinear: LessWrong Debate on Steepening AI Progress Curves — teortaxesTex · 2026-11-08
- Reviewers reward flaw-free work, users reward useful work — why peer review punishes innovation — wzenus · 2026-11-08
- Math community debates Glazer's proof and OpenAI's H10 paper: slop need not compound — littmath · 2026-11-08
- Stanford paper: more info helps some LLM trading agents, wrecks others — rohanpaul_ai · 2026-11-08
- Neel Nanda pushes back on LRH critique: one steering vector, four norms, four behaviors — NeelNanda5 · 2026-11-08