Meta-model writes weight updates from one paragraph, beating LoRA on many benchmarks
gdm3000 · x · 2026-11-17
- Roi Cohen et al. trained a meta-model that reads a one-paragraph task description and directly writes the weight update for Qwen3.5-2B in seconds on a small GPU, with no gradient step.
- Across 75 public benchmarks, the generated update improved 59 and hurt 8: ComVE jumped 50.0→91.1, three-object logical deduction 46.4→82.0; GSM8K (final-answer-only) rose 5.7→18.6, beating a LoRA trained on up to 1,400 GSM8K rows (12.7).
- With the same 16 labeled examples, it beat LoRA on 24 benchmarks and lost on 14. Framed as an early attempt at "compressing backpropagation into a model."
More from Research
- Re-Search benchmark shows LLMs can spot good research ideas but can't generate them — Muhan Gao · 2026-11-17
- BrickFlow uses SE(3) flow models to reconstruct buildable LEGO assemblies from images — Peter Kulits · 2026-11-17
- ReDiverse fixes diversity collapse in distilled diffusion models via activation steering — ReDiverse · 2026-11-17
- SPARE preprint: 2.55x faster decoding for linear attention on a 1.3B model — krishnan · 2026-11-17
- Study estimates ~12% of philosophy published today may be AI-generated — leland_mcinnes · 2026-11-17
- Berkeley EECS & Statistics open all-rank faculty search in AI for Physical Sciences — berkeley_ai · 2026-11-17