Training a Model for $60 Beats Claude 2 on SWE-bench
StanfordAILab · x · 2026-08-14
A developer turned Karpathy's nanochat into a SWE-bench speedrun, demonstrating that high-performance coding models can be trained at extremely low costs.
Experiment results:
- $60 in compute: Achieved a 5.0% pass@1 rate on SWE-bench, comparable to the 2023 SOTA model Claude 2.
- $1000 in compute: Achieved an 11.0% pass@1 rate, surpassing Claude 3 Haiku.
Starting from randomly initialized model weights, this experiment proves that with an optimized training pipeline, the coding capabilities of early frontier models can be replicated or even surpassed at a fraction of the compute cost.
Related event: Researchers Train SWE-bench Model for $60, Rivaling Claude 2(2 posts)→
More from Research
- Quiz: Double the Residual Stream from 4096 to 8192 — How Does Param Count Change? — Sauers_ · 2026-10-16
- Researcher credits GPT for multiple accepted papers, incl. disproving Wegner's 1965 conjecture — ctjlewis · 2026-10-16
- Open-source shopper attention heatmap: YOLO11-pose + Gaze-LLE + Depth Anything V2 — claudioe · 2026-10-16
- AutoAgent0: VLM-powered agentic runtime lifts long-tail driving success by 28.3pp — yining_hong · 2026-10-16
- Shuffled labels hit AUC 0.59: selection bias, not signal, drives prosody benchmarks — maier_ak · 2026-10-16
- CODESTRUCT reframes codebases as AST action spaces to fix brittle LLM code agents — burkov · 2026-10-16