Training-Free Weight Editing at 3ms: Decision Model Distillation Matches LoRA
Nandakishor_ml · reddit · 2026-11-10
Developer Nandakishor built on his open-source decision model Laya to directly edit LLM weights without training: a closed-form distillation of an energy-based decision engine (fitted on 300 rows) into Qwen 3.5 0.8B, teaching it skills like timely tool calling—no gradient descent. Benchmarks on 93 held-out requests show tool-choice accuracy 80%→93.5%, ready yes/no 18%→97.8%; 87s total on 2×T4 with the edit itself taking 3ms, changing only 262K weights (vs 263M for LoRA); WikiText perplexity +10% vs LoRA's +15%; only 1 of 15 unrelated prompts damaged, no catastrophic forgetting. Code is on Kaggle.
More from Research
- Researcher says Claude offered a roadmap to prove his long-sought math result — miniapeur · 2026-11-10
- Ant Group's SkinMind: agentic RAG with graph probing for dermatology AI — antgroup · 2026-11-10
- New training losses make ViTs develop specialized spatial attention heads — Yuji Nozawa · 2026-11-10
- DeepMind's AlphaProof Code Formally Verifies Signal Protocol in a Month — sebkrier · 2026-11-10
- Shaun Gallagher dissects how "robodiment" differs from human embodiment — AnnaCiaunica · 2026-11-10
- Sillage 3D: open-source browser engine renders Gaussian splats as brush strokes — janusch_patas · 2026-11-10