Hutter Prize text compression jumps ~10% by overfitting small transformers for arithmetic coding
ricklamers · x · 2026-11-19
TacoCohen highlights a 10% recent leap in the Hutter Prize for text compression — a huge trend break. The approach trains/overfits a small transformer on the data to be compressed, so it assigns high likelihood (and short arithmetic-coding code lengths) to tokens; since the receiver doesn't know the model, its compressed weights must be counted in the code length, so training loss includes that cost. Rick Lamers adds that swarms grinding on compression have effectively infinite training data and an un-hackable reward, and will keep paying dividends.
More from Research
- Google's GUARD paper: frozen base model, self-editing agent harness gains +8.89 points — omarsar0 · 2026-11-19
- Kezek splits mono two-person conversations into per-speaker tracks for full-duplex dialogue training data — manifestro · 2026-11-19
- Elie Bakouch: current AI research benchmarks overfit and can't track RSI — eliebakouch · 2026-11-19
- HybridApexW optimizer: 2x faster than torch.compile AdamW+Muon baseline — omendUUde · 2026-11-19
- Sandia Labs' VASt model predicts high-entropy alloy properties from tens of samples — CurieuxExplorer · 2026-11-19
- FoodTruck Bench, an LLM agent business-sim benchmark, lands on Papers with Code — NielsRogge · 2026-11-19