Autoresearch Pretraining Feature on nanochat Scales to Thousands of GPUs With Auto Recovery

SeunghyunSEO7 · x · 2026-11-19

A new pretraining feature enables autoresearch on Karpathy's nanochat scaling ladder — running on a few nodes or launching jobs across thousands of GPUs with automatic recovery. Reposter Seunghyun Seo floats a 'scaling law speedrun' idea: a competition where participants submit their power-law fits and the winner is whoever scales better — with an illustrative plot showing Muon/Shampoo + Dense reaching a target loss at 2x the compute efficiency of AdamW + Dense.

Original post →

More from Infra

Infra channel →