Berkeley and DeepMind paper: LLM representational geometry comes from symmetry in language statistics
udmrzn · x · 2026-11-20
UC Berkeley and Google DeepMind researchers show that the striking geometries inside LLM representations — circular months, rippled timelines of years, linearly decodable city maps — don't emerge from transformers inventing world models. Even word2vec produces the same shapes. The geometry derives from translation symmetry in pairwise word co-occurrence statistics, which the authors formalize to analytically predict representational manifolds that match empirical results and remain robust to perturbations.
More from Research
- TMol puts Rosetta-style all-atom energies in PyTorch, 350x throughput on batched H200s — ArashVahdat · 2026-11-20
- AWS architect's independent benchmark: Snap 1.2 is the only order-invariant decision model — ryanallred · 2026-11-20
- AtlasKV: billion-scale knowledge graphs in 20GB VRAM, no retriever or vector DB needed — udmrzn · 2026-11-20
- OnchainBench launches: benchmarking whether crypto AI agents can be trusted with your keys — DavideCrapis · 2026-11-20
- Fig previews RoleCUA: 780+ hours of expert work to train agents for month-long job functions — adityaag · 2026-11-20
- Realtime-Bench: humans win games 72% vs 32%, but models crush complex web tasks — ManlingLi_ · 2026-11-20