New 'Epiplexity' concept rethinks information for computationally bounded AI — and eval design
burny_tech · x · 2026-10-10
An arXiv paper, "From Entropy to Epiplexity" by Marc Finzi, Shikai Qiu, Yiding Jiang, Pavel Izmailov, J. Zico Kolter and Andrew Gordon Wilson, argues Shannon information and Kolmogorov complexity fail to capture how much a computationally bounded learner can extract from data, surfacing three paradoxes: deterministic transforms can't add information, information is order-independent, and likelihood modeling is mere distribution matching.
The authors introduce epiplexity — the structural, learnable content of data for bounded observers, excluding time-bounded entropy from sources like PRNGs and chaotic systems — and show information can be created through computation. Daniel Fein highlights its relevance to eval design: models should fail in ways that "aren't stupid," meaning they miss knowable structure rather than randomness.
More from Research
- EMNLP paper: diffusion LMs internally encode latent denoising timestep representations — gsarti_ · 2026-10-23
- KU Leuven and VIB open two Group Leader professorships in AI for Biology, start 2027 — jpillowtime · 2026-10-23
- Subspace Tuning curbs LLM catastrophic forgetting by freezing principal singular directions — scottniekum · 2026-10-23
- Perceptic launches PercepticOS, an AI workspace for 'scientific superintelligence' — nathanbenaich · 2026-10-23
- EPFL's Bosselut to talk long context as test-time learning at ELLIS seminar — andre_t_martins · 2026-10-23
- DRATS: Adaptive Task Sampling Boosts Worst-Task Performance in Multi-Task RL — Cohere · 2026-10-23