Pure RL Trains a Strong 4B Continual-Learning LLM, No Distillation or SFT
aviral_kumar2 · x · 2026-11-13
aviralkumar2 reports training a very strong 4B LLM using only RL with proper rewards on the right curricula—no distillation or SFT—achieving a much better tradeoff between continual learning ability and performance.
On the Continual Learning Bench codebase adaptation task, the model outperforms all reported systems, trailing only the team's latest frontier models. The shared leaderboard shows their Qwen3.5-4B rising from 1.33 (untrained, ACT+WRITE loop) to 10.55 cumulative reward (+2.27 gain), rivaling GPT-5.4 ICL (10.14) but still below frontier systems like Claude Opus 5.5 (17.40) and GPT-6 Astra (17.25).
More from Research
- Mathematician uses AI to rebuild Euclid's windmill proof of Pythagoras, sans triangle cut — AlexKontorovich · 2026-11-14
- Blind 3-provider LLM benchmark: GPT-6.1 Sol beats Claude 9-0, structured output dialects differ — WorthRow3022 · 2026-11-14
- Chinese Paper Maps CoT's Hidden Topology: Reasoning Behaves Like a Molecule, Not a Line — mikeflache · 2026-11-14
- 1.26M blind votes show users judge AI more harshly each month — models aren't declining — rohanpaul_ai · 2026-11-14
- EmbodiedSWE wins CoRL 2026 Best Paper: coding agents generate robot training data — PeterHndrsn · 2026-11-14
- Largest systematic agentic eval campaign launches: test agents as tool users, not tools — animesh_garg · 2026-11-14