CMU-led team proposes "epiplexity," a new information measure for computationally bounded learners
davidmanheim · x · 2026-10-27
Researchers from CMU and NYU (Marc Finzi, Shikai Qiu, J. Zico Kolter, Andrew Gordon Wilson, et al.) released an arXiv paper, "From Entropy to Epiplexity: Rethinking Information for Computationally Bounded Intelligence."
Core argument: Shannon information and Kolmogorov complexity assume observers with unlimited computation, so they come up nearly empty-handed when quantifying the learnable content of data. The authors identify three apparent paradoxes:
- Deterministic transformations cannot increase information — yet computation in practice clearly creates learnable content;
- Information is independent of data ordering — yet curriculum effects show order matters;
- Likelihood modeling is merely distribution matching — yet it can yield programs more complex than the data-generating process.
They introduce "epiplexity," formalizing what computationally bounded observers can learn from data: structural content minus time-bounded entropy (the unpredictable content of PRNGs and chaotic systems). This framework shows how information can be created by computation, why data ordering matters, and how likelihood modeling can produce more complex programs than the generating process itself.
More from Research
- Music Lenses: ISMIR paper uses sparse autoencoders for steerable concept-level music retrieval — umpedronosapato · 2026-10-27
- Neuronpedia adds 7M+ neurons for Qwen3-32B, Gemma and Llama 3.3 70B to its interpretability encyclopedia — aryaman2020 · 2026-10-27
- Matthew Green hails stunning new ETH result pending Lean proof validity — matthew_d_green · 2026-10-27
- Heuristic for prediction accuracy: a bimodal probability histogram peaking at 0 and 1 — moultano · 2026-10-27
- AI agent autonomously discovers and implements high-order Maxwell solver scheme — burny_tech · 2026-10-27
- OpenAI claims its AI invented a new math technique, cracking a 35-year-old wave-focusing problem — 141_1337 · 2026-10-27