NanoGPT Speedrun record shattered: 39.9s to 6.7s with retrieval-augmented n-gram hints
tatsu_hashimoto · x · 2026-10-29
Herman Brunborg improved the NanoGPT Speedrun record from 39.9s to 6.7s (-83.2%), a bigger gain than every record since 2025 combined (-81.4%). Stanford professor Tatsunori Hashimoto called the improvements "completely absurd" and recalled Herman dominating nearly every CS336 leaderboard.
The technique: retrieval-augmented n-gram hints to handle rare, repeated web content that LMs can't memorize in time. Hashimoto also noted the counterintuitive point that nanoGPT's big gains came from AI-assisted humans rather than autoresearch. The attached chart is the Artificial Analysis Intelligence Index v5.0 leaderboard spanning nine evaluations, with Gemini 3, Claude Opus 4.5, and GPT-5.2 near the top.
More from Infra
- Agents push compute back to CPUs: $221B server CPU market seen by 2030 — BenBajarin · 2026-10-29
- AI server revenue forecast to hit $920B in 2027, pushing market past $1T — Beth_Kindig · 2026-10-29
- Oracle deploys Google TPUs for the first time and scales Vera Rubin NVL72 clusters to multi-gigawatt scale with Nvidia — Beth_Kindig · 2026-10-29
- Google Cloud expected to post 95.1% Q3 growth, outpacing AWS and Azure — Beth_Kindig · 2026-10-29
- lium rents NVIDIA B300s at $4/GPU-hour, H200 prices for Blackwell Ultra — bittingthembits · 2026-10-29
- Microsoft Expected to Generate $8B+ Cash Flow in Q3 While Meta, Google, Amazon Burn AI Capex — Beth_Kindig · 2026-10-29