Why reasoning distillation fails on 9B models: six LoRAs, mixed results
SignificantZebra5883 · reddit · 2026-11-13
A developer training a Slovak legal agent on Qwen shares detailed results from six fresh LoRAs. The same recipe that got a 27B PoC near Opus 5 doesn't transfer cleanly to 9B: thinking lost on the legal-CPT model (10:20), tied on stock chat (15:15), and won on stock base (18:12), while every 9B variant still loses badly to 27B (best 7:23) with notably higher fabrication scores.
Setup: 500M-token legal CPT corpus, 4-bit QLoRA (rank 64, alpha 128, LR 2e-4), two SFT stages (1199 + 400 examples, teacher CoT 1,000 tokens median), evaluated on 30 held-out Slovak court decisions with Opus 5 judging randomized A/B pairs.
Caveats the author flags: tiny eval set, Opus is both teacher and judge, and the 9B (Qwen3.5) vs 27B (Qwen3.8) mismatch means model size isn't isolated. Open questions: is 9B just too small, does CoT need far more examples to overtake no-thinking, and is there an efficient way to teach reasoning without distillation? Long-term plan: apply the recipe to a future 27B Qwen4 inside a Claude Code-style harness with search and RAG.
More from coding & agent
- JoMoHa jointly RL-trains model and harness, hits 97.0% on ALFWorld — omarsar0 · 2026-11-13
- YC F26 team open-sources runtime to run 10,000 agents in one session — KlausCodes · 2026-11-13
- Open-Source Pipeline Turns LiDAR + OpenStreetMap into Isometric Pixel-Art Cities with FLUX.2 — zsakib_ · 2026-11-13
- X, MIT, UW & Stanford Launch WSDM Cup for AI Agents Writing Community Notes, $10k Prizes — NathanpmYoung · 2026-11-13
- Benchmarks won't move users off Claude Code — Antigravity needs real product uplift, dev argues — Justin_Halford_ · 2026-11-13
- Why AI labs must own meeting software: spoken context is the missing fuel for agents — signulll · 2026-11-13