Liquid AI post-training head: SFT lifts pass@k, RL sharpens it to pass@1
ziv_ravid · x · 2026-11-18
Maxime Labonne, Head of Post-Training at Liquid AI and author of the LLM Course, on why post-training now defines models.
- Traces the evolution from SFT and preference optimization to RL with verifiable rewards, and now multi-domain on-policy distillation (MOPD): separate teachers for code, math and other domains merged back into one student
- His intuition: SFT and distillation raise pass@k, and RL sharpens it into pass@1; you still can't skip SFT, and stage order matters
- Explains why RL works even though a long rollout yields roughly one bit of signal
- Covers training small models for agentic tasks directly in the harness, which open-source RL stacks are production-ready, looping transformers, and how much big-model work could move to local small models
- Claims models write worse than a year ago because labs chase agentic benchmarks, and that RL environment companies are overrated
- Also discusses safety, regulation, open source and Chinese models, recursive self-improvement, and whether Europe can build frontier models
Related event: Liquid AI Post-Training Lead Details Two Years of SFT/RL/MOPD Shifts(2 posts)→
More from Models
- Indie dev's model scans raw binaries directly, scoring 0.977 on code blocks — wavefnx · 2026-11-18
- OpenAI engineers secretly use Astra for Law to write PRDs and design docs — sherwinwu · 2026-11-18
- Astra 6.2 is likely an improved checkpoint, not Bel — insider speculates Bel lands as 6.5 — haider1 · 2026-11-18
- Microsoft launches seven MAI models at Ignite, matching GPT-6 quality for under 2% of the cost — marlene_zw · 2026-11-18
- Google insider: Gemini CUA now just 0.9% behind OpenAI's just-released model — allenainie · 2026-11-18
- mitsuhiko on GPT-6.2 Astra/Sol: 'feel whelmed' after trying both — mitsuhiko · 2026-11-18