Basalt: open-source Blackwell inference engine hits 665 tok/s, 2.6x Strata on a 5090 rig
jesdga95 · reddit · 2026-10-10
Basalt is a heavily tuned, MIT-licensed Strata fork targeting Qwen3.8 Flash-Next on Blackwell GPUs. It reaches 665 tok/s structured and 354 tok/s prose on a 5090+5060 Ti (2.6x Strata), 8-user concurrency at 623 tok/s total, a custom vision encoder 3-4x faster than llama.cpp, and single-file repacked weights with OpenAI/Anthropic-compatible APIs.
More from coding & agent
- Brian Holt sold his 42U homelab — a Framework Desktop plus coding agents does more — film_girl · 2026-10-10
- Group-Evolving Agents wins COLM 2026 award, hits 71% on SWE-bench Verified — xwang_lk · 2026-10-10
- Asking an agent to fix a bug you don't understand is continuous paperclip maxxing — brandon_xyzw · 2026-10-10
- exe.dev offers SSH-first cloud sandboxes for developers and AI agents, drawing rave reviews — davidcrawshaw · 2026-10-10
- Surge AI launches sudo L7: a benchmark testing whether coding agents can act like staff engineers — rmcwhorter99 · 2026-10-10
- Live feed shows what images AI agents use while hunting for new planets — BLUECOW009 · 2026-10-10