NanoGPT training record smashed to 9.65 seconds using clever CPU co-processing
charles_irl · x · 2026-10-03
A relay of open-source PRs has driven the nanoGPT speed training record from 39.9 seconds down to 9.65 seconds. @hermanbrunborg first used a longest exact match model running concurrently on CPU to cut the record to 21.5s; DevenPzak generalized the CPU approach to a larger share of tokens to reach 9.65s, building on work from nthngdy, cyrusghane and Daniel Monroe. The author notes CPU usage is a gray area in nanoGPT rules and all records remain subject to review.
More from Research
- DNA sequence watermarks are 'more theater than security', says Stanford's Anshul Kundaje — anshulkundaje · 2026-10-04
- Meta paper: RL post-training hurts test-time scalability — the 'Sharpening Tax' — dair_ai · 2026-10-04
- Review Papers Without Strong Opinions Are Dead, Says Researcher; AI Slashes Data-Collection Work — jwt0625 · 2026-10-04
- AutoCompact trains agents to compact context themselves, +9.2 on SWE-bench Verified — omarsar0 · 2026-10-04
- ICLR 2027 trims review scores to a 4-point scale, reviewers puzzled — random-tomato · 2026-10-04
- EPFL paper recasts multi-view stereo as seq2seq, beating MVS and feed-forward baselines — CSProfKGD · 2026-10-03