Could 'already clean' neural nets enable training-free continual learning via VSA memory?
RepresentativeRice63 · reddit · 2026-10-18
Digesting Nick Cammarata's Goodfire essay "Neural Networks Are Already Clean", a Redditor highlights four findings: neurons act as bipolar control-flow switches; the weakest 75% of activations can be zeroed outside early layers with little accuracy loss; foundational circuits recur across models (Qwen3-32B and Claude 3.5 Haiku converge on the same 4-neuron hub for preeclampsia); memorized facts sit at the output and route through existing circuits (clipping 99.9th-percentile activations in Qwen3.6 erases verbatim recitation but keeps reasoning).
The speculative proposal: encode facts as sparse ternary recipes stored as hypervectors in a Vector Symbolic Architecture, letting a small frozen model query external memory—adding or deleting facts without retraining. Open obstacles include early-layer continuous signal strength, entanglement with semantic routing, and the need for a cross-model coordinate registry.
More from AGI Musings
- AI researcher: in the agent era execution is cheap and good taste is the scarce resource — rose_e_wang · 2026-10-18
- Blogger predicts AI will fully dominate chip design within two years, like it did math — rakyll · 2026-10-18
- Luiza Jarovsky warns AI is creating a society cognitively enslaved to model owners — LuizaJarovsky · 2026-10-18
- Nathan Lambert: no evidence open weights lead AI cyberattack damage — natolambert · 2026-10-18
- Economist debate: AI-era labor shifts to human-valuable work, total labor spend won't fall — herbiebradley · 2026-10-18
- Paying per token to move a robot would be hilarious — and exactly what OpenAI and Anthropic want — philfung · 2026-10-18