3Blue1Brown: compressing text to 1 bit per character with arithmetic coding

3Blue1Brown · youtube · 2026-11-18

3Blue1Brown releases Part 3 of the "Compression is Intelligence" series, explaining how arithmetic codes push text toward 1 bit per character and what that says about prediction, pretraining, and generalization. The video references Ilya Sutskever's observation on generalization, prequential coding, the Hutter Prize, and ANS literature, with chapters covering Shannon's estimate, reframing pretraining, and arithmetic codes in action.

Original post →

More from Models

Models channel →