Shannon's 1950 Hand-Calculated English Entropy: 80% First-Guess Accuracy with 100-Letter Context
alexbilz · x · 2026-08-06
Recalling Claude Shannon's classic 1950 experiment to measure the entropy of the English language. His wife served as the subject, guessing a Jefferson biography text letter by letter. With 100 letters of context, her first guess was accurate 80% of the time. From this, Shannon bounded the entropy of English between 0.6 and 1.3 bits per letter, concluding that the language is roughly 75% redundant.
More from Research
- OPD-V: Optimizing Visual Reasoning via Modality Balance as Privileged Info — Aniri · 2026-08-06
- Meta Unpacks Multimodal Pretraining: Strong Generation with 5% Compute — facebook · 2026-08-06
- UniWorld-View: Large-Baseline Novel View Synthesis via Video Diffusion — Haiyang Zhou · 2026-08-06
- ToolArtist: Agentic Image Generation via Unified Multimodal Models — Jiahao Zhao · 2026-08-06
- HelloWorld: Enabling Real-Time Social Interaction in Video World Models — Liangyang Ouyang · 2026-08-06
- TriGlue: A Biology-Inspired Generative Framework for Molecular Glue Design — Yuliang Yan · 2026-08-06