Lightning Weave composes accuracy and efficiency specialists for +9.7pts with 20% fewer tokens
songhan_mit · x · 2026-10-14
Lightning Weave, a post-training framework from MIT and NVIDIA, composes capabilities from independently post-trained specialists — an accuracy shift (Klear) and an efficiency shift (DECS) — distilled at every token of the student's own reasoning, producing models that are both more accurate and shorter.
Key results (Qwen3.5-4B):
- Average accuracy +9.70 pts over five benchmarks (55.16 → 64.86).
- HMMT 2025: 59.2 → 64.0 with 10.7% fewer tokens; LiveCodeBench v5: 41.7 → 54.2 with 9.6% fewer.
- Qwen3-4B response tokens drop from 15,723 to 12,628 (−19.7%).
- All five students tested achieve the best accuracy-efficiency sweet spots vs. base and single-anchor policies.
- 1.84× fewer GPU-hours than online DOPD: a tilted target stabilizes offline training, each anchor pair scores data once, then the student trains alone.
More from Research
- SURGE Surgical Foundation Model: Perception Saturates After Just One Labeled Video — maier_ak · 2026-10-14
- Prompteina: First Text-Conditioned All-Atom Protein Structure Generation Model — PangWeiKoh · 2026-10-14
- Correction Training frees diffusion models from CFG as looped architectures gain traction — CSProfKGD · 2026-10-14
- FramePack-U: Training-free constant-memory frame packing boosts long-video VLM benchmarks by up to 12.9% — UNC-ChapelHill · 2026-10-14
- CineRecon Benchmark: 12 Frontier Models Struggle to Reconstruct Editable 3D Scenes from Video — hkuhk · 2026-10-14
- Meituan's ClawLoop: Self-Improving Agents via Teacher-Free Harness-Native Training Loops — meituan-longcat · 2026-10-14