Autoresearch Pretraining Feature on nanochat Scales to Thousands of GPUs With Auto Recovery
SeunghyunSEO7 · x · 2026-11-19
A new pretraining feature enables autoresearch on Karpathy's nanochat scaling ladder — running on a few nodes or launching jobs across thousands of GPUs with automatic recovery. Reposter Seunghyun Seo floats a 'scaling law speedrun' idea: a competition where participants submit their power-law fits and the winner is whoever scales better — with an illustrative plot showing Muon/Shampoo + Dense reaching a target loss at 2x the compute efficiency of AdamW + Dense.
More from Infra
- KAIST Proposes PROLA: Low-Rank Adaptation for Spatiotemporal Weather Downscaling — MLVLab · 2026-11-19
- Bypassing PyTorch with HIP kernels makes MiniMax H3 up to 3x faster on AMD R9700 — Apprehensive_Sky892 · 2026-11-19
- Ex-DeepSeek core member open-sources Linger, a local AI agent claiming 5x faster inference — Aiden_Tech_Ai · 2026-11-19
- China opens first overseas-targeted AI cluster on Huawei Ascend chips — pstAsiatech · 2026-11-19
- Tencent says US chip curbs 'unfortunate', may slow AI training short-term — pstAsiatech · 2026-11-19
- Samsung dismisses AI bubble fears: neoclouds are just 8% of the market, chip shortage to persist — BenBajarin · 2026-11-19