0.8B model beats 2B on ARC-Challenge (42.15%) via closed-form weight surgery with zero backprop
AdventurousTwo6445 · reddit · 2026-10-06
A Reddit user presents DynamicTune: treating deep networks as continuous-depth dynamical systems, extracting a velocity flow from a larger teacher and transplanting it into a student via closed-form linear algebra (SVD deltas) — zero backprop, zero training tokens, 12 minutes on an 8GB AMD RX 580. To rule out local eval bias, the unquantized FP16 checkpoint was uploaded to Hugging Face and independently benchmarked by TPN Bench on a datacenter NVIDIA L4 with lmeval 0.4.12.
Full ARC-Challenge (1,172 questions, zero-shot, greedy):
- Stock Qwen3.5-0.8B Base: 37.50%
- SFT distillation (25k Claude pairs, 3-epoch multi-GPU): 38.10%
- Stock Qwen3.5-2B Base (Q8): 41.10%
- DynamicTune surgery: 42.15% (+4.65% over baseline, Z = 3.23σ, p < 0.001)
Key technique: applying deltas to all 24 layers blows up NLL (+64.78%), so the authors compute normalized Shannon spectral entropy of representation residuals per layer to select receptive layers (Layer 0, H = 0.7138) and skip high-entropy ones. The result is author-claimed plus third-party benchmarking, not peer-reviewed, but the pipeline (public checkpoint, standard eval framework, coordinator run ID) is verifiable.
Related event: Closed-form weight surgery transplants 4B model capabilities into 0.8B(2 posts)→
More from Models
- Claude's speech glitch contaminated the whole model ecosystem via rampant distillation — PMinervini · 2026-10-06
- Cognition's SWE-2 outperforms Fable-5.1 and Opus 5.5 on Animation Bench, testers report — himanshustwts · 2026-10-06
- Grok 4.7 sweeps top 3 on VulcanBench Frontier v4 repo-level coding benchmark — XFreeze · 2026-10-06
- Dev runs every task in both Codex and Claude Code: Opus 5.5 beats GPT-6 Astra — JeremyNguyenPhD · 2026-10-06
- User Pushes Back on GPT-6 Astra Efficiency Claims, Says Claude Opus 5.5 Limits Feel More Generous — CtrlAltDwayne · 2026-10-06
- Benchmarks vs reality: local quantized Qwen3.8 27B beats cloud flash next in real coding workflows — SeriousJul · 2026-10-06