Indie run: 23k tps decode on a single RTX 5090, matching Llama 3.2 1B at ~90% less cost per token
jon_durbin · x · 2026-10-03
Developer jondurbin shares preliminary results from his Kappa training run (Gated DeltaNet-2, 576B checkpoint, 9T tokens): it beats Llama 3.2 1B on ARC-C, OBQA, TQA and nearly matches it on ARC-E, SciQ, at roughly 90% lower cost per token. His Parallax vLLM fork hits 23k tps decode at high batch on a single RTX 5090 (28k with fp16 GDN2) and 70k tps prefill; llama.cpp fork, mobile inference and training code will be open sourced soon. Known issues include an underflow in a GDN2 per-channel decay gate.
Related event: Open Pretraining Matches Llama 3.2 1B at One-Tenth the Cost(3 posts)→
More from Infra
- Anthropic's compute spend grows 4x/year, $10B in Q3 alone, set to overtake OpenAI by May 2027 — scaling01 · 2026-11-19
- Thailand data center association cites $18B in GPU/hardware cancellations after SE Asia lending halt — SumitGup · 2026-11-19
- Gridline launches one-API fine-tuning, distillation and batch inference for open models — Scobleizer · 2026-11-19
- Ascend output could hit ~20M chips next year; memory and packaging, not logic, are the bottleneck — teortaxesTex · 2026-11-19
- AMD Loom boosts Qwen3.5-4B prefill up to 2.6x on Radeon 890M laptops — jfowers_amd · 2026-11-19
- Dev submits PR adding periodic checkpoints to GKE sandboxes so agents don't lose progress — yoimnotkesku · 2026-11-19