NanoGPT trained in 39.9 seconds, a new world record cutting 46% of training time

cephaloform · x · 2026-09-29

Deven Pietrzak (MIT Math, Hyperstition lab) set a new NanoGPT training record: 39.9 seconds (0.665 minutes), claiming the percentage reduction in training time beats the previous 45 world records combined. The PR (#360) to Keller Jordan's modded-nanogpt was merged after 5 days of work and independently reproduced by four separate sources.

Key techniques:

This cut 34 seconds (46%) of training time on the same hardware versus the previous record.

Related event: NanoGPT speedrun record slashed to 39.9 seconds(4 posts)→

Original post →

More from Infra

Infra channel →