Tuning 0.001% of Gemma 3 270M's weights lifts sort accuracy from 1% to 97%
dejanseo · x · 2026-11-02
A replication of the NeurIPS 2025 study "Quantifying Elicitation of Latent Capabilities in Language Models" (Anthropic, UC Berkeley, Thinking Machines, Scale AI et al.) shows that training a tiny number of weights can unlock skills no prompt can reach.
- The original study, across Llama 3 and Qwen 2.5 models (1B–8B) on ten tasks, found 10–100 trained weights recover up to 50% of the gap to full fine-tuning, and thousands recover 95%.
- On Gemma 3 270M, the best of 20 prompt wordings got 5-number sorting right only 1% of the time; training 1,000 of its 100M weights (0.001%) pushed accuracy to 97%.
- The same method enabled word-list reversal (84%) and clean JSON output (99% with just 30 weights), and improved long addition — but multiplication stayed near 30% even with 100,000 trained weights.
- Skills the model already half-knows need orders of magnitude fewer weights, so the count needed reveals how latent a capability is.
More from Models
- Reflection to host Reddit AMA on Nov 3 about Beam, its first open-weight model — Aggressive-Sign-8167 · 2026-11-02
- Cursor Pro+ user burns through all quotas, says forced to upgrade to $200 Ultra plan — pramodk73 · 2026-11-02
- User regrets $500 OpenAI plan: 'ultra fast' is actually buggy computer use — imjustnewatai · 2026-11-02
- Users report Astra 6 quality collapse: endless apologizing loops, worse than models from a year ago — Icy-Way3920 · 2026-11-02
- GLM-5.3 repeatedly violated 'no memorized code' rule in security evals, Irregular finds — shoumikgoswami · 2026-11-02
- SemiAnalysis launches GDPx benchmark: best model Claude Opus 5.5 scores just 12% — zephyr_z9 · 2026-11-02