Open Pretraining Matches Llama 3.2 1B at One-Tenth the Cost

Developer jondurbin shared results of the open-source 'Kappa run' pretraining: at a 576B checkpoint trained on 9T tokens, it matches Llama 3.2 1B benchmarks at roughly one-tenth the cost, alongside inference code reaching 23k tps on a single RTX 5090.

2026-10-03 ~ 2026-10-04 · 3 related posts