Local inference shootout: Basalt beats Strata at 118 tok/s on Qwen3.8 with half the RAM
x_MASE_x · reddit · 2026-11-10
Reddit user xMASEx benchmarked two local inference engines, Strata (CLI 0.1.59) and Basalt (0.3.2), on the same model — Qwen3.8-Flash-Next IQ3S with 256k context — on a 12900K / 128GB DDR4 / RTX 5070 Ti box.
Strata: 99.2 tok/s average decode, prefill 839; 15.3GB VRAM but a hefty 69.2GB RAM. Generation stuttered when switching between phases, compaction broke, and tasks often didn't finish.
Basalt: 100–118 tok/s average, peaking at 145; prefill peaks 4200; 15.8GB VRAM and only 53GB RAM. Smooth generation, correct compaction, far fewer errors, and it kept working until the project was done.
The author calls it the first local setup since Qwen-3.6-35B that actually feels reliable, and plans to test nvfp4 quantization next, plus Swift and DeepSeek.
Related event: Real-World Test: Basalt Beats Strata by 20% in Speed and Stability on 128GB(2 posts)→
More from Infra
- Nvidia's biggest Southeast Asian partner imported far more AI chips than Nvidia saw in its data centers — nmasc_ · 2026-11-11
- AMD hits all-time high as traders hedge NVIDIA risk in AI buildout — BenBajarin · 2026-11-11
- Rain launches Agent Vaults to embed agentic payments into any app — ai · 2026-11-11
- Google broadens TPU lineup with two new specialized chips for confidential training and bigger memory — TechCrunch AI · 2026-11-10
- vLLM-Omni benchmarks: 2048px image generation in 0.63s on 4x B300 — vllm_project · 2026-11-10
- ReCaVSR one-step streaming 4x video super-resolution ported to ComfyUI — LatentSpacer · 2026-11-10