Autonomous Agents Beat Human Baseline in nanoGPT Speedrun Experiment
eliebakouch · x · 2026-08-16
Prime Intellect conducted a massive autonomous agent research experiment using Codex (GPT 5.5) and Claude Code (Opus 4.7) to optimize the nanoGPT training speedrun. After 10k runs and 14k H200 hours, Opus set a new record of 2930 steps, beating the human baseline of 2990. The study reveals that agents excel at hyperparameter sweeps but struggle with novel ideas, and traces the breakdowns in autonomy. All data has been open-sourced.
More from coding & agent
- How do you debug bad agent runs when you already have tracing? — Sensitive-Parsnip-12 · 2026-10-16
- Where does pre-task context live for agents? Devs debate workflows beyond handoff — shishen2026 · 2026-10-16
- No-instruction autonomous agent wakes up, sets own goals, and tries hacking by default — sethlazar · 2026-10-16
- CODESTRUCT reframes codebases as AST action spaces to fix brittle LLM code agents — burkov · 2026-10-16
- Would you install another SDK for agent debugging on top of your tracing stack? — Sensitive-Parsnip-12 · 2026-10-16
- Law student builds all-in-one AI study platform LexLab Studio with ChatGPT and Muse — Acrobatic_Rhubarb598 · 2026-10-16