Hutter Prize text compression jumps ~10% by overfitting small transformers for arithmetic coding

ricklamers · x · 2026-11-19

TacoCohen highlights a 10% recent leap in the Hutter Prize for text compression — a huge trend break. The approach trains/overfits a small transformer on the data to be compressed, so it assigns high likelihood (and short arithmetic-coding code lengths) to tokens; since the receiver doesn't know the model, its compressed weights must be counted in the code length, so training loss includes that cost. Rick Lamers adds that swarms grinding on compression have effectively infinite training data and an un-hackable reward, and will keep paying dividends.

Original post →

More from Research

Research channel →