AI Agent Autonomously Writes Triton Kernel, Breaking NanoGPT Speedrun Record
cong_ml · x · 2026-08-05
An AI agent has set a new world record of 75.4 seconds in the NanoGPT Speedrun challenge.
Achieved by @congml and @RecursiveSI, the breakthrough relies on an autonomously found solution featuring a custom ReLU^2 MLP Triton kernel. During the forward pass, it only saves relu(X@W1)^2 and reconstructs relu(X@W1) during the backward pass as a fused epilogue. This optimization significantly reduces memory traffic, shaving 0.6 seconds off the previous record.
More from coding & agent
- Enterprise Agent Permissions: Approve Every Action or Batch by Risk? — Any-Economics8950 · 2026-08-05
- Open-Source Agent That Calls You for Decisions When It Gets Stuck — RichardsonDx · 2026-08-05
- Fixing Harness Issues Doubles Agent Scores: A New Engineering Course — rajistics · 2026-08-05
- Fixing Lost Agent Context Skyrockets Model Evaluation Scores — rajistics · 2026-08-05
- Gary Marcus: AI Agents Are Wildly Unreliable, Need SMEs to Babysit — GaryMarcus · 2026-08-05
- Programmer Spends Thousands on AI Subscriptions: Generates 1M Words in 5 Hours — vista8 · 2026-08-05