20 minutes on one H200: GRPO post-training makes Qwen 3.5-2B more accurate and token-efficient
johnolafenwa · reddit · 2026-09-04
Open-source models tend to over-reason on every problem. The author released a notebook and video implementing GRPO from scratch and using it to post-train Qwen 3.5-2B for better accuracy and reasoning efficiency.
The interesting result: trained purely on simulating a Python interpreter, the model became substantially more accurate and token-efficient on math problems — a clear generalization effect. Training took 20 minutes on a single H200, and the code applies to any open-source model.
- Code: agi-playground/grpo
- Full walkthrough and results: YouTube video
- Before/after result charts included
More from Research
- Spherical Fixed-Point Reasoners: keeping looped transformers bounded and teaching them when to stop — GabGarrett · 2026-10-21
- KAIST & Meta's RPL: programmable retrieval via RL makes a 4B model beat all 8B baselines — CShorten30 · 2026-10-21
- Perplexity ships Decider V1.2, tops Vals AI's new ClaimBench decision-model benchmark — bo_wangbo · 2026-10-21
- Masking giveaway terms barely dents LitGene's gene-relation scores, study shows — bravo_abad · 2026-10-21
- Omnii-1 genome model debuts: first AI-designed phages ever given to a human patient — JosephJacks_ · 2026-10-21
- How LLMs store facts: three learning phases, attention circuits, and hallucinations — scott_linderman · 2026-10-21