MIT dissects Anthropic's J-space: averaged Jacobian keeps under 10% of variance
burny_tech · x · 2026-10-29
Responding to Anthropic's J-space paper, MIT's @neurostrow and the Fiete group analyzed what the Jacobian lens behind Claude's 'global workspace' actually captures: the lens averages over many prompts and positions, and in four open models the averaged Jacobian lands far from the cloud of individual ones. What survives averaging are directions that grow through layers, stay stable across text, and behave identically across contexts—which is why J-space covers under 10% of activation variance. The authors care about the same question at @mostikai: which parts of a model's internal state are stable enough for another model to read.
More from Research
- OmniSpectra: a wavelength-native spectral foundation model that skips destructive resampling — Md Khairul Islam · 2026-10-29
- Qwenputer compiles an exact 32-bit RISC-V CPU into Qwen3.5-4B weights, with 83.3% multi-session recall — solyarisoftware · 2026-10-29
- LIMe cuts LLM training compute by 15% with layer-integrated attention memory — ChampionshipLimp1749 · 2026-10-29
- Cohere Labs Presents 3 Papers at EMNLP 2026 ORACLE Workshop on Culture and Multilinguality — Cohere_Labs · 2026-10-29
- Cohere's Zero-Shot SLAM-ASR Spoken Language Understanding Paper Featured at EMNLP — Cohere_Labs · 2026-10-29
- AReA: training-free attention method pushes Qwen Image and Flux to 6K — linoy_tsaban · 2026-10-29