spaCy creator deconstructs the "LLM writes weights" paper: it detects loops, not mental simulation
honnibal · x · 2026-11-03
honnibal (spaCy/Explosion founder) clarifies what the "LLMs natively read & write neural network weights" paper actually does:
- No mental simulation: the model runs the network, notices the output is looping, then devises a weight intervention — inspecting activations, adjusting weights, and rerunning. Backpropagation would work fine here too, but the paper apparently wants to claim it doesn't use backprop.
- Game experiment: Astra authors weights to play a game by first devising a rule-based AI, then encoding that rule into the weights.
- Meta-observation: research abstracts increasingly feel like stage magic — each iteration uses a different trick, so you can't assume methods repeat across papers.
A valuable thread for puncturing abstract-level hype around the study.
Related event: spaCy founder dissects LLM weight read-write research(2 posts)→
More from Research
- Old Vision Architecture Tricks Like ResNets Are Being Reinvented Weekly, Says Researcher — code_star · 2026-11-03
- Cohere Labs talk: changing model architecture late in training instead of freezing after pretraining — Cohere_Labs · 2026-11-03
- Quiz: Can You Diagnose All Four Training Scenarios From Their Loss Curves? — cneuralnetwork · 2026-11-03
- NVIDIA, Tsinghua, CMU, Berkeley and Princeton launch StreamEval, first real-time interactive streaming video benchmark — JaynitMakwana · 2026-11-03
- MIT: AI edits 0.016% of another model's weights, lifting MATH-500 from 7.9% to 48.1% — rohanpaul_ai · 2026-11-03
- Schmidhuber: AI weight editing is 'old hat' from his 1990s fast weight programmers — SchmidhuberAI · 2026-11-03