Simplex recovers telescoping-cone belief geometry from LLM activations
Hidenori8Tanaka · x · 2026-09-10
Simplex (Astera Institute) published 'The geometry of nonergodic composition' by Kyle J. Ray, Paul M. Riechers, and Adam S. Shai:
- Core question: LLM pretraining data is heterogeneous across many sources; this 'nonergodicity' implies a specific computational structure for next-token prediction
- Theory: if a network represents beliefs linearly, belief updates over multi-source data should form 'telescoping cones'—per-source components whose magnitude scales with contextual evidence
- Empirics: the authors recover these telescoping cones via linear regression from a transformer's residual-stream activations, with components growing and shrinking in line with in-context evidence
It extends the team's earlier work on linear belief representation in LLMs, making implicit Bayesian inference explicit as measurable activation geometry.
Related event: Simplex Finds 'Telescoping Cone' Belief Geometry in LLM Activations(3 posts)→
More from Research
- Santa Fe Institute opens 2027 Complexity Postdoctoral Fellowships, deadline Sep 30, 2026 — yoavartzi · 2026-09-11
- Are Lean soundness bugs a real risk? Weighing how AI proofs translate to ZFC — jessi_cata · 2026-09-11
- Lean AI proofs to ZFC: researcher says translation is feasible, with caveats — jessi_cata · 2026-09-11
- Novel Reasoning Effort Control Scheme Analyzed: Graded GRPO Training — stochasticchasm · 2026-09-11
- Building a full LLM from scratch over weekends with Stanford's CS336 — stanfordnlp · 2026-09-11
- Universal YOCO paper combines recursive compute with efficient attention for depth scaling — donglixp · 2026-09-11