Neural Thickets: adding noise to LLM weights boosts reasoning for free — it's just sampling in disguise
kastnerkyle · x · 2026-10-31
A repost of nicodufour's thread on Neural Thickets (@yulegan, @phillipisola): inject random noise into an LLM's weights, keep the best copies, and let them vote — reasoning improves with no training at all.
The follow-up explains why it works: it's sampling in disguise. Swapping noise on billions of weights for a single random vector on the logits yields the same gains, suggesting the benefit comes from decode-time randomness rather than the weight perturbations themselves.
More from Research
- Revisiting why transformers can't do zero-shot recursion, via Transformer Circuits math — akbirthko · 2026-10-31
- AI-assisted proof: first high-probability regret bounds for Feel-good Thompson sampling — QuanquanGu · 2026-10-31
- Vitória Piai's free psycholinguistics book Speaking joins ChapterPal's open-access library with an AI tutor — burkov · 2026-10-31
- Neuron Lens: A database of every neuron and attention head in Qwen3-8B and Gemma 4 12B — CatAstro_Piyush · 2026-10-31
- You can fingerprint your exact GPU with a single torch.index_add_ call — CatAstro_Piyush · 2026-10-31
- 500bp jitter needed to kill center bias in genomic sequence models, says Kundaje — anshulkundaje · 2026-10-31