Agent ran 89 experiments to improve a small model — 92% of gains came by experiment 44
ccerrato147 · x · 2026-09-29
In one run of Karpathy's autoresearch pipeline, an agent ran 89 experiments to improve a small model. By experiment 44 it had already captured 92% of the total gain; the machine kept working while returns kept shrinking. The takeaway: automating the research loop doesn't automatically accelerate improvement.
More from AGI Musings
- David Patterson: AGI will replace knowledge and physical work at the same time — davidpattersonx · 2026-09-29
- Blogger Proposes RLEI, a Replacement for RLVR Based on Epistemic Incompleteness — ryunuck · 2026-09-29
- As Models Surpass Human IQ, Alignment May Mean Answers Humans Understand — djcows · 2026-09-29
- JevBench Creator: Picking Models by 'Vibes and Trust' Is a Bad Joke — airesearch12 · 2026-09-29
- KOL predicts AI reaches flawless 'Artificial Universal Intelligence' by 2028-2030 — davidpattersonx · 2026-09-29
- Blogger reframes the singularity: not machines passing humans, but humans surrendering moral judgment — AryHHAry · 2026-09-29