Synthesizing CoT training data with a FIM model to train 2B/9B reasoning GANs
cephaloform · x · 2026-09-04
The author shares a simple data-synthesis technique for enabling tasks a model can't yet do reliably, used for a reasoning discriminator in a 2B GAN: sample a ton of model attempts, train a FIM (fill-in-the-middle) model to complete the CoTs within those attempts, fill in reasoning for a set of gold-star answers, then SFT on the synthetic chains and run RL.
They also applied the same recipe to a 9B model used for both generator and discriminator: the generator's SFT set takes ground-truth completions from real texts, with reasoning chains filled in by the FIM-trained model.
More from Research
- Anima Anandkumar's first podcast on Accelerated Understanding: 5T context and 4D physical AI — AnimaAnandkumar · 2026-09-05
- Uno pairs AR weights with diffusion LoRA adapters, beating EAGLE-3 and all diffusion LLMs — HongyiWang10 · 2026-09-05
- Self-explanation training generalizes beyond narrow hint formats to held-out evals — a_karvonen · 2026-09-05
- Two training targets from behavior investigations: counterfactual predictions and open-ended self-explanations — a_karvonen · 2026-09-05
- Anthropic Fellows train models to explain their own wild behaviors with generalization to held-out evals — a_karvonen · 2026-09-05
- Deep Learning Weekly #471: Claude Fable 5.1 launch, production-parity LLM evals, alignment paper — dl_weekly · 2026-09-05