Forking returns in post-training replay; freezing the n-gram table helps
ZimingLiu11 · x · 2026-10-27
In part 5 of the thread, Ziming Liu shows that post-training also replays data, and forking reappears on small code, math, and RLVR-style corpora. Freezing the n-gram table helps: charts comparing table-trainable vs table-frozen across ID, Magicoder, OpenR1, and RLVR-MATH show the frozen table markedly shrinks the train/validation loss gap.
Related event: Explaining LLM Forking: Mechanism and Mitigation(2 posts)→
More from Research
- Physicist lets Claude Opus pick its own research question — result validated by experts — IgorCarron · 2026-10-27
- FlexGlue: sparse-to-dense matching that beats dense accuracy at a fraction of the cost — zhenjun_zhao · 2026-10-27
- Mundo turns a few room photos into an explorable, simulatable 3D world — at 1.5 FPS — mark_k · 2026-10-27
- Mathematics as an experimental science: Marc Lelarge to talk LLM-driven math experiments at ENS — marc_lelarge · 2026-10-27
- DOE Genesis Mission Phase II: AI for particle accelerators and rare isotope separation — prescott · 2026-10-27
- Tenstorrent runs Proteina protein model at high throughput, claims no HBM shortage — gajesh · 2026-10-27