TraceForge generates family-native reasoning traces, boosting Qwen3.5 pass@1 by up to 7.1 points
jiqizhixin · x · 2026-11-03
Independent researcher Yan Wang introduced TraceForge, which trains on generated reasoning traces written in the student model's own style instead of copying a teacher model's hidden reasoning.
- Generator + critic adversarial training: the generator learns to reconstruct family-native traces from context and answers, while a critic distinguishes real from generated traces; five rounds of adversarial co-training refine both and support candidate selection.
- Results: on Qwen3.5-9B-Base with identical teacher answers, swapping Fable 5's traces for generated family-native traces improves final-checkpoint HumanEval+ and MBPP+ pass@1 by 3.7 and 7.1 percentage points.
- Key principle: keep style, truth and evidence separate—an authentic-looking trace isn't necessarily a correct one.
More from Research
- Team Replicates Anthropic's "Teaching Claude Why" Alignment Method on Qwen and Open-Sources It — austinc3301 · 2026-11-03
- LoRA From Scratch: recovering a hidden weight change with two thin matrices — dima806_dima · 2026-11-03
- Contextual AI ships final contextual embedding models: 0.6B sharing 9B's embedding space — antoine_chaffin · 2026-11-03
- DLR-Lock proposes making open-weight LLM fine-tuning far costlier without breaking inference — burkov · 2026-11-03
- Free 30-min lesson: check if your model's 0.8 really means 80% — bgoncalves · 2026-11-03
- RaiSim 2.13.0 lets you run robotics examples online and adds a fast fluid dynamics model — ChongZzZhang · 2026-11-03