Open Pretraining Run Matches Llama 3.2 1B at One-Tenth the Cost

Developer jondurbin shared early results from the Kappa pretraining run, matching Llama 3.2 1B benchmarks at roughly one-tenth the cost, with inference code achieving 23k tokens/sec decoding on a single RTX 5090.

2026-10-03 ~ 2026-10-03 · 2 related posts

• Open pretraining run matches Llama 3.2 1B on ARC-C at ~10% of the cost, author details 4 pitfalls — jon_durbin · 2026-10-03
• Indie run: 23k tps decode on a single RTX 5090, matching Llama 3.2 1B at ~90% less cost per token — jon_durbin · 2026-10-03