3Blue1Brown: compressing text to 1 bit per character with arithmetic coding
3Blue1Brown · youtube · 2026-11-18
3Blue1Brown releases Part 3 of the "Compression is Intelligence" series, explaining how arithmetic codes push text toward 1 bit per character and what that says about prediction, pretraining, and generalization. The video references Ilya Sutskever's observation on generalization, prequential coding, the Hutter Prize, and ANS literature, with chapters covering Shannon's estimate, reframing pretraining, and arithmetic codes in action.
More from Models
- Google Unveils Gemini 4 Hydrogen to Power Task Execution in Gemini — GeminiApp · 2026-11-18
- Top token user vouches for Carbon: strong at xhigh for complex coding — olcan · 2026-11-18
- Anthropic S-1 reveals models older than six months earn under 20% of revenue — every · 2026-11-18
- Open source AI isn't dead: one Qwen model pulled 6M Hugging Face downloads last month — alex_verem · 2026-11-18
- vLLM ships DSpark draft model for Kimi K3.1 Pro with up to 30% higher acceptance length — vllm_project · 2026-11-18
- Cloudflare's Clairic: open-ended extraction with confidence scores, no preset categories — thursdai_pod · 2026-11-18