DeepMind's 'Language Modeling Is Compression': Chinchilla 70B beats PNG and FLAC
anpaure · x · 2026-11-17
anpaure adds the theoretical backbone to the discussion — DeepMind's paper Language Modeling Is Compression: predictive models and lossless compressors are interconvertible, making LLMs powerful general-purpose compressors.
Key results: text-only-trained Chinchilla 70B compresses ImageNet patches to 43.4% of raw size (beating PNG's 58.5%) and LibriSpeech audio to 16.4% (beating FLAC's 30.3%). The compression lens also yields new insights into scaling laws, tokenization, and in-context learning, and lets any compressor (e.g., gzip) act as a conditional generative model.
More from Research
- Sharpa's SALA aligns latent actions across human and robot video for dual-axis scaling — Jiachun Jin · 2026-11-17
- OutfitterBench tests LLM agents on 200-day rental business, exposing commitment failures — Jie Huang · 2026-11-17
- Craftax framework evolves code policies via feedback cascade, reaching final stage 50% of time — Víctor Gallego · 2026-11-17
- NervePool: A simplicial pooling layer built on the nerve complex, published with open code — Pseudomanifold · 2026-11-17
- Connectomics nonprofit E11 Bio is hiring across imaging, ML and wet-lab roles — anderssandberg · 2026-11-17
- Semantic Cache Pitfalls: Should LLM Caching Drop Dense Embeddings Entirely? — Artyrred · 2026-11-17