Basalt 0.2.0: 300 t/s prose at higher-than-llama.cpp quality on a single RTX 5090 rig
jesdga95 · reddit · 2026-11-04
The author released Basalt 0.2.0, a Blackwell-only open-source inference engine for Qwen 3.8 Flash Next. On a 5090 + 5060 Ti 16GB + 128GB RAM rig it hits 300 t/s prose, 600 t/s structured output, and >10,000 t/s prefill at 64K context, with 262K context and 4 parallel slots. Quality vs BF16 reference: llama.cpp Q8 scores KLD 0.0325 / 93.98% top-1 match, while Basalt Q8 scores 0.0200 / 95.34% — beating the llama.cpp baseline. A new NVFP4-DQ8 build trades 7% speed for KLD 0.0561 vs 0.0624 on DQ6K. All lossy hacks are off by default with warnings. Speed gains come from a faster draft step, output head, and expert placement; the .basalt format is now v2 with a migration tool. The repo has 500 stars.
More from Infra
- Nina Schick: we are industrializing superintelligence — intelligence-per-watt is the new power law — NinaDSchick · 2026-11-04
- Cerebras CTO: our plate-sized chip drew more power than an entire 5 kW rack — AccBalanced · 2026-11-04
- AMD guides 67% revenue growth but Nvidia's growth floor next year is 70% — BenBajarin · 2026-11-04
- China launches 18 satellites at under $1.5M each as Spacesail constellation grows to 274 — heyshrutimishra · 2026-11-04
- Analyst: AMD's $17B revenue guidance is likely the floor, not the ceiling — BenBajarin · 2026-11-04
- AMD surges 6% after-hours on strong AI-driven earnings, adding $50B in market cap — econoar · 2026-11-04