797GB of CPU-side n-gram lookups match 57% more compute on 1B LM training
PangWeiKoh · x · 2026-10-30
Researchers tested whether n-gram counts in read-only CPU RAM can help LM training. Scaling to 797GB of n-grams, they matched 57% more compute on a 1B-parameter model; a similar trick recently broke nanoGPT speedrun records. Paper linked in the thread.
More from Infra
- Intel down 23% post-earnings: AI server demand booms while PC business collapses — tengyanAI · 2026-10-30
- Arizona asks residents to limit showers to 30 seconds while AI data centers drink its water — CtrlAltDwayne · 2026-10-30
- Cloudflare developer users grow 10x in 3 years, from 1M to 10M — yihui_indie · 2026-10-30
- By 2028, 'country of geniuses in a datacenter' will be limited only by datacenter count — timshi_ai · 2026-10-30
- Redditor jokes about finding racks of idle GPUs at a local data center — Beneficial-Ad-8127 · 2026-10-30
- Wafer hits 2 trillion daily tokens on OpenRouter, doubling from 1T in just 4 days — ycombinator · 2026-10-30