Language statistics symmetry shapes LLM representational geometry (ICML 2026)
burny_tech · x · 2026-10-27
burnytech shares the arXiv paper "Symmetry in language statistics shapes the geometry of model representations" (ICML 2026). It proves translation symmetry in language co-occurrence statistics governs representational geometry — months as circles, years as smooth 1D manifolds, city coordinates linearly decodable — with analytic predictions matching real embedding models and LLMs, and shows the geometry is robust to statistical perturbations due to an underlying latent variable.
Related event: Paper: Language Symmetry Shapes Representation Geometry(2 posts)→
More from Research
- Insilico's AI-designed Pan-TEAD inhibitor ISM6331 hits 40% ORR in mesothelioma Phase 1 — rand_longevity · 2026-10-27
- ETH result stuns complexity theorists: what structure could all boolean circuits share? — moultano · 2026-10-27
- RL doesn't just reward thinking longer—it incentives parallel agent swarms — rosstaylor90 · 2026-10-27
- Reversed audio has identical magnitude spectra: the time-frequency resolution tradeoff — techNmak · 2026-10-27
- Researcher argues DoM steering is no more transparent than gradient-based vector selection — johnhewtt · 2026-10-27
- DoM steering working at a localized layer doesn't prove localization is correct — johnhewtt · 2026-10-27