797GB of n-grams in read-only CPU RAM match 57% more compute at 1B-scale LM pretraining
anshulkundaje · x · 2026-10-16
The COUNT project trains LMs on top of empirical n-gram count tables precomputed into read-only CPU RAM — 797GB of n-grams. At 1B scale this matches 57% more compute, after a similar trick recently broke nanoGPT speedrun records. The tables are easy to precompute and store, significantly speed up pretraining, and can be hot-swapped at inference time for training-free domain adaptation.
Related event: 797GB n-gram Table in CPU Memory Boosts Pretraining Compute by 57%(2 posts)→
More from Infra
- Local AI is a routing problem, not a privacy feature: where each task should run — bigdata · 2026-10-16
- Moving agent work from GPUs to CPUs for durable, predictable actions — cramforce · 2026-10-16
- First TPUv7 agentic benchmark on vLLM claims 1.42x perf-per-dollar over GB300 NVL72 — vllm_project · 2026-10-16
- Phison CEO calls CXMT memory scare a 'ghost story', warns of materials shortage — BenBajarin · 2026-10-16
- Ben Bajarin: The Future of Compute Is Fungible as Hyperscalers Design for Workload Flexibility — BenBajarin · 2026-10-16
- AirLLM runs frontier-size models layer-by-layer: 671B on 12GB VRAM — Aiden_Tech_Ai · 2026-10-16