Closed-form weight surgery transplants 4B model capabilities into 0.8B

Researchers propose a closed-form 'weight surgery' method that transfers capabilities from a 4B teacher to a 0.8B student using only a few calibration prompts and SVD-based weight updates, with no backpropagation. The resulting 0.8B model reaches 42.15% on ARC-Challenge, outperforming 2B models.

2026-10-06 ~ 2026-10-06 · 2 related posts

• Closed-form weight surgery transfers 4B capabilities into 0.8B with 4 anchor blocks — AdventurousTwo6445 · 2026-10-06
• 0.8B model beats 2B on ARC-Challenge (42.15%) via closed-form weight surgery with zero backprop — AdventurousTwo6445 · 2026-10-06