TraceForge: Distilling with Your Own Thinking
Yan Wang
cs.AI, cs.LG
2026-10-21
On the same 5,000 questions and the same Fable 5 answers, traces rewritten in the student family's voice raise Qwen3.5-9B by 3.7 pass@1 on HumanEval+ and 7.1 on MBPP+.
Frontier models now often return an answer or a short summary. The reasoning trace, the writing the model does before the answer, stays hidden. Distillation uses that writing as supervision. An answer alone is a thin label.
Replaying a provider's encrypted reasoning blocks across models can recover the hidden text. That breaks the API terms, so it is a poor base for a product. TraceForge leaves the teacher's trace alone. It reads the context and the teacher's answer, then writes a new trace in the student family's own voice that is supposed to arrive at that answer. That trace is called family-native.
The generator has a narrow job. It sees the system prompt, the user turns, any tool calls, and the final answer, and it fills in the trace the family would have written. The loss sits only on that trace. The answer is already in the input, so the model practices the derivation that ends there.
Supervision is about 10,000 Qwen3.8-Max conversations with the real family trace known. No closed teacher is in that data. The generator is a Qwen-family model, so the voice is learned from examples rather than requested in a prompt. Pointed at a new teacher answer, it writes a derivation that lands on that answer. It is not replaying a trace it was shown in training.
Supervised training is not close enough. An outside judge, GPT-6.1-SOL, scores how likely a trace is to be a real family trace, from 0 to 1. Real traces average 0.66. After this stage the generator averages 0.54.
A critic learns to tell real traces from generated ones. Each of the 10,000 samples gets one generated negative, and training stops once the real trace is at least 0.2 more probable than that negative, rather than stretching the margin as far as it will go. A smaller critic failed. Within a few hundred steps it fit shortcuts, scored well on held-out pairs, and left the generator with nothing to learn. The critic has to be at least as large as the generator.
Five rounds of online co-training come next. Each round the generator writes 5,000 traces. Traces that fool the critic become more critic data. Traces that fail become preference pairs, and the generator is updated with DPO, a preference update on one better sample and one worse sample. The judge stops rising after five rounds. The generator alone scores 0.68, a bit above the 0.66 of real traces. The critic's choice among four candidates scores 0.82. These numbers measure how real a trace looks. Correctness is measured separately, on benchmarks.
The generator and the critic never train on Claude or OpenAI tokens. The teacher appears only when a trace is requested, and only as context plus an answer.
Every student is full-parameter SFT from Qwen3.5-9B-Base. The pool is 5,000 Claude Code trajectories at a 256K context, eight H20 GPUs, one fixed recipe. Only the source of the trace and the answer changes. One run per cell. ARC-Easy, ARC-Challenge, and BoolQ test general ability. HumanEval+ and MBPP+ test coding with pass@1, the fraction solved by a single sample. At about 1,600 packed-batch updates:
| Corpus | HumanEval+ | MBPP+ |
| S1 Fable 5 trace + answer | 64.63 | 52.38 |
| S2 family trace + Fable 5 answer | 68.29 | 59.52 |
| S3 Qwen3.8-Max trace + answer | 46.95 | 24.60 |
| S4 TraceInversion + Sol answer | 3.66 | 6.61 |
| S5 family trace + Sol answer | 59.15 | 60.05 |
On the three multiple-choice tasks the five corpora stay within 1.6 points of each other, and none leads all three. Those items never use a reasoning trace.
S2 and S1 share Fable 5's answers and differ only in who wrote the trace. At the end that swap is worth 3.7 on HumanEval+ and 7.1 on MBPP+. On MBPP+ S2 leads at steps 300, 600, 900, 1,200, and 1,600. HumanEval+ has an exception: at step 600, S1 is at 66.46 and S2 at 56.10.
S1 is unstable. HumanEval+ reaches 66.46 at step 600, falls to 33.54 at step 1,200, and recovers to 64.63. The step-1,200 evaluation was run twice more, and the score held.
S5 and S4 share GPT-6.1-SOL's answers. TraceInversion, an existing method that reconstructs a trace from the answer, writes in a generic voice, uses numbered steps this family does not write, and often restates the answer. It peaks at 21.95 and 42.59 at step 900, then finishes at 3.66 and 6.61. S5 finishes at 59.15 and 60.05. At step 300 its HumanEval+ is 0.61, and every later checkpoint is higher. Averaged across the two coding sets, S5 finishes just ahead of S1.
S3 finishes about 21 and 35 points behind S2, with the answers changed as well, so it is not a style-only comparison. Late in training, S3 and S4 pick up dead loops and repetition. Fable 5's own traces and the family-native traces do not.
With the questions and the Fable 5 answers held fixed, a trace in the student's family voice is the better coding teacher, and the steadier one. S2 climbs to 68.29 and 59.52. S1 drops more than 30 points in the middle and has to climb back.
Qwen3.8-Max is the strongest model in the family whose terms allow distillation. Training on its own traces and answers still loses. Teacher strength sets the ceiling. A trace written the way the student already writes decides how much of that ceiling shows up in the student. The gain is on coding. The multiple-choice scores barely move.
A stricter pipeline hands the family trace to Qwen3.8-Max, lets that model write the answer, and trains on the pair. Reported performance matches S2, and no Claude or OpenAI token enters the backward pass. The numbers for this variant are not in the tables.
One run per cell. S1 moves by more than 30 points on HumanEval+. The extra 3.7 at the end is a step-1,600 result. At step 600 the ranking flips.
The 21- and 35-point gaps against Qwen3.8-Max mix answer quality with trace style. S1 against S2, and S4 against S5, show downstream gaps. They still do not separate family style from every other difference between the pipelines. The related-work section says this outright.
The scores 0.68 and 0.82 are GPT-6.1-SOL's probability that a trace looks genuine. No human rated them, and they do not measure whether the reasoning is right. Before the adversarial rounds the generator still trailed real traces, 0.54 against 0.66.
Generator and critic sizes are not reported, aside from the critic having to be at least as large. The student is a single 9B base. The 5,000 trajectories are confidential. S5's slow start is explained by an unchecked claim treated as widely said: Qwen3.5 mid-training saw a lot of Claude-family data and little GPT-family data. Less catastrophic forgetting is not measured. Dead loops in S3 and S4 are guessed to come from distilling an already distilled model, a habit called breathing second-hand smoke. No rate is counted.
The front matter once says 3,500 questions. The experiment section says 5,000 trajectories. A table note calls S4 the strongest of the five at step 900, but Table 2 has S2 at 66.46 and 59.52 there, and S4 at 21.95 and 42.59.