Basalt: A fast Blackwell inference engine for Qwen 3.8 Flash Next with dynamic vision
jesdga95 · reddit · 2026-10-13
A Reddit user open-sourced Basalt, an inference engine forked from Strata and heavily tuned for Blackwell GPUs, for 3.8 Flash Next.
- Vision support normally requires keeping the image encoder in VRAM (evicting experts) or CPU (slow)
- New --vision dynamic mode keeps the encoder in RAM at idle (0 VRAM at load), uploads it to VRAM in 30ms on demand, then refills expert slots from RAM in 60ms
- No image sent = zero performance hit on text
- Custom kernels make encoding up to 3x faster than llama.cpp
More from Infra
- AMAT declines WFE forecast; analyst's own chart shows WFE going "to the moon" — BenBajarin · 2026-10-13
- Applied Materials CEO: AI drives WFE growth, yield innovation adds 10ppts margin by CY'30 — BenBajarin · 2026-10-13
- Huawei passes 100M Kirin-powered devices, tests next-gen Kirin 2027 chip — pstAsiatech · 2026-10-13
- 3090 FE VRAM hits 112°C in seconds running LLMs — power cap benchmarks inside — Farmadupe · 2026-10-13
- One-button KVM swap between PC and Mac — and remote wake for your AI agent — noahsolomon · 2026-10-13
- Cache-busting bug masked a 24x GPU gap: 11 confident falsehoods, none from the model — colinmcnamara · 2026-10-13