DeepMind's 'Language Modeling Is Compression': Chinchilla 70B beats PNG and FLAC

anpaure · x · 2026-11-17

anpaure adds the theoretical backbone to the discussion — DeepMind's paper Language Modeling Is Compression: predictive models and lossless compressors are interconvertible, making LLMs powerful general-purpose compressors.

Key results: text-only-trained Chinchilla 70B compresses ImageNet patches to 43.4% of raw size (beating PNG's 58.5%) and LibriSpeech audio to 16.4% (beating FLAC's 30.3%). The compression lens also yields new insights into scaling laws, tokenization, and in-context learning, and lets any compressor (e.g., gzip) act as a conditional generative model.

Original post →

More from Research

Research channel →