Twin neuralese is interpretable: linear translator decodes hidden-state code words with 93% accuracy
Sauers_ · x · 2026-11-01
Building on @Sauers' neuralese work, @salinecrop showed that hidden-state communication between twin models (Qwen3-4B and AFM-4.5B) is interpretable. Each model holds a private code word, reading the other's hidden states at 8 of 36 layers. A linear translator fit on 183 other words names the passing code word correctly on 93% of 80 unseen words (logit lens: 8%, chance: 1.25%), meaning an outside reader can decode what passes between the twins.
More from Research
- LessWrong rebuttal argues SAE 'sanity check' paper's toy setup is hostile, not easy — burny_tech · 2026-11-01
- LeCun and Balestriero prove the Platonic Representation Hypothesis: JEPA views beat scaling with 42x fewer samples — burny_tech · 2026-11-01
- BCI wearables could turn brain arousal signals into implicit personal rewards for AI tuning — burny_tech · 2026-11-01
- Study: Syntax evolved by gradually repurposing pre-existing sensorimotor circuits — abenitezburraco · 2026-11-01
- Eric Michaud on the purpose of interpretability: a deep science of what nets learn and why — burny_tech · 2026-11-01
- Neel Nanda et al. reverse-engineer grokking, showing it emerges in three continuous phases — burny_tech · 2026-11-01