Lightning Weave: Improving the Accuracy-Efficiency Frontier of Reasoning Models through Capability Composition
Yecheng Wu, Song Han, Han Cai
cs.AI
2026-09-14
Lightning Weave composes accuracy and efficiency policy shifts into one student with Tilted-Target DOPD. Qwen3.5-4B HMMT 2025 rises from 59.2% to 64.0% with 10.7% fewer tokens.
Reasoning models often buy accuracy with long traces. Cut the traces and exploration dies. Length-penalized RL, mode distillation, and weight-space merges all try to hold both ends. Joint optimization still fights itself: the accuracy objective wants more search, the efficiency objective wants earlier stops.
Specialists trained separately already exist. Klear-Reasoner-8B is the accuracy specialist; DECS-1.5B is the short-answer specialist. Lightning Weave, from MIT and NVIDIA, asks whether those independently acquired behavioral changes can be extracted and composed into one student, instead of co-optimizing two opposing rewards from scratch.
The object of transfer is the policy shift from post-training, not the specialist's endpoint policy.
Each capability is an anchor pair: the checkpoint before a post-training stage and the specialist after it. The token-level log-ratio of their probabilities is the behavioral change that stage introduced. That ratio strips out habits the base model already had.
Three steps.
Naive cached DOPD fails as the student leaves the behavior policy: the KL regularizer estimated from cached actions no longer matches the intended penalty, so gradients can stay non-zero at the desired solution. Tilted-Target writes the destination down. Loss and gradient vanish when the student equals q. After scoring, training serves only the student.
Set one weight to 1 and the rest to 0 and the objective collapses to the matching single-anchor run. Sweeping weights traces different accuracy-efficiency operating points.
The main setup composes Klear (Qwen3-8B-Base to Klear-Reasoner-8B) with DECS (DeepSeek-R1-Distill-Qwen-1.5B to DECS-1.5B) into five students. Benchmarks: AIME 2024/2025, HMMT 2025, LiveCodeBench v5/v6. Math is Avg@64, code is Avg@4. AES is computed versus each student's own base.
| Student | Avg acc (base to weave) | Avg tokens | AES |
| Qwen3-1.7B | 34.66% to 37.94% | 16,375 to 15,052 | 0.34 |
| Qwen3-4B | 55.90% to 59.92% | 15,723 to 12,628 (-19.7%) | 0.41 |
| Qwen3-4B-Thinking-2507 | 64.47% to 68.94% | 19,863 to 18,084 (-9.0%) | 0.28 |
| Qwen3.5-4B | 55.16% to 64.86% (+9.70) | 25,439 to 24,086 (-5.3%) | 0.61 |
| OLMo-3-7B-Think | 56.38% to 60.12% | 20,557 to 17,901 (-12.9%) | 0.34 |
Headline numbers: Qwen3-4B AIME 2024 moves from 72.9% to 76.0% with 21.3% fewer tokens. Qwen3.5-4B HMMT 2025 moves from 59.2% to 64.0% with 10.7% fewer tokens; LiveCodeBench v5 from 41.7% to 54.2% with 9.6% fewer tokens. Across all five backbones, composition beats both single-anchor AES scores.
Accuracy-only lengthens traces. Efficiency-only often drops below the base. Composition puts both shifts on the same tokens.
On Qwen3-4B-Thinking-2507, ρDECS=0.625 hits 84.9% AIME 2024 at 15,048 tokens, ahead of SimpleSD (76.7% / 19,473), Art (78.5% / 15,456), PromptCoT 2.0 (75.8% / 20,177), and weight interpolation MI (83.1% / 15,885). Non-dominated weave points form an empirical Pareto curve.
Ablations on Qwen3-4B: Tilted-Target AES is close to online DOPD (0.41 vs 0.38 on two anchors) without serving four live models. Naive offline lags (Klear-only AES -0.15 vs 0.23). Data mixture AES is 0.29. Sequential composition is order-sensitive: Klear then DECS scores 0.31, the reverse 0.37. Weave gets 0.41.
Adding MiMo-RL as a second accuracy anchor: Klear+MiMo-RL reaches 64.38% average accuracy but grows tokens from 15,723 to 18,577. Three-anchor mix posts AES 0.42.
If a team already has an accuracy checkpoint and a short-answer checkpoint, this is cheaper than another multi-objective RL run: one student rollout, one scoring pass, then student-only training. Weights are a knob. Different Pareto points do not each need a fresh RL job.
It also shows post-training deltas can be added in token-level policy space without matched architectures. Different tokenizers are handled by projection. The spirit is close to task arithmetic, applied to policies rather than weights.
This is a composition recipe, not a new reasoner. Token savings stay smaller than efficiency-only methods. The bet is accuracy that does not collapse. That trade is useful for 4B-7B deployments that watch both score and bill, less so for maximum truncation.
The authors list three. All students are under 10B; scale-up is untested. The method needs matched pre/post anchors, which new tasks may not have. Efficiency gains remain smaller than methods trained only to be short.
A few more. Each configuration is trained once; eval repeats measure decoding noise, not seed variance. Cached rollouts cap at 2,048 tokens while math eval goes to 32,768 and code to 40,960, so the student learns on short traces and is tested on long ones. Code is "coming soon." AES punishes accuracy drops harder than it rewards gains, so the ranking favors "do not regress"; that is a legitimate design choice, but AES numbers do not travel cleanly across papers.