NanoGPT Speedrun Frontier: 153 Autonomous Runs Across 18 Models Open-Sourced
samsja19 · x · 2026-08-16
PrimeIntellect released full results and traces from the NanoGPT optimizer speedrun benchmark. The evaluation covers 153 autonomous runs across 18 frontier models, including GPT-5.6, Claude, DeepSeek V4, and Kimi K3. The leaderboard tracks best validation loss achieved within 24 hours, percentage of closed test cases, and execution trajectories. All tool calls and generated files are open-sourced for reproducibility.
More from coding & agent
- Devs say LLM coding mistakes are a different class: overcomplication, inferred limits, worse at higher reasoning — Mu5_ · 2026-10-16
- How do you debug bad agent runs when you already have tracing? — Sensitive-Parsnip-12 · 2026-10-16
- Where does pre-task context live for agents? Devs debate workflows beyond handoff — shishen2026 · 2026-10-16
- No-instruction autonomous agent wakes up, sets own goals, and tries hacking by default — sethlazar · 2026-10-16
- CODESTRUCT reframes codebases as AST action spaces to fix brittle LLM code agents — burkov · 2026-10-16
- Would you install another SDK for agent debugging on top of your tracing stack? — Sensitive-Parsnip-12 · 2026-10-16