Viewpoint: Optimizing for language compression led to current AI level
kuchaev · x · 2026-08-29
Christian Szegedy comments that optimizing for language compression is a profound and surprising result that led to the current level of AI. The cited content notes that Shannon estimated English is 75% redundant, with entropy of ≤1 bit per letter given long-range context. Modern LLMs compress English text to under 1 bit per character due to accurate prediction.
More from Research
- Co-Scientist evaluation: Severe hallucinations drop to 4%, fabrication to 0% — SRSchmidgall · 2026-08-29
- AI Scientist limitations: Selective reporting and code-method divergence remain — SRSchmidgall · 2026-08-29
- Gemini Designs Precursor for MXene Synthesis in New Materials Science Experiment — SRSchmidgall · 2026-08-29
- Co-Scientist deployed across materials science, biology, and CS with varying autonomy — SRSchmidgall · 2026-08-29
- Google releases Co-Scientist to accelerate real-world scientific discovery — SRSchmidgall · 2026-08-29
- CarNet Matches Spherical-Harmonics Accuracy in Cartesian Space for Atomic Simulations — bravo_abad · 2026-08-29