antirez: DwarfStar runs Qwen3.8 Flash Next on 64GB Macs at 50-70 tok/s with SSD n-grams
antirez · x · 2026-09-14
antirez announced that DwarfStar now supports Qwen3.8 Flash Next, with the work done by @ivanfioravanti in a PR.
- Performance: runs well on 64GB Mac systems with 50-70 tok/s inference and >1400 tok/s prefill; Metal-only for now.
- Technique: uses the same SSD-backed n-grams approach as DS4.1F to offload n-gram caches to disk.
In a follow-up he notes the setup may or may not make sense on the 128GB DGX Spark (extra speed plus free RAM are tempting), but is clearly worthwhile for ROCm users with 64GB Framework Desktops.
More from Infra
- Your AI agent would cost 260% more without prompt caching, Arena data shows — arena · 2026-10-30
- Nebius runs auction-driven GPU pricing, lifting rates ~20% amid tight supply to 2028 — BenBajarin · 2026-10-30
- macOS update silently re-adds eMudhra root certs users had removed — steipete · 2026-10-30
- Bittensor's Terza Inference routes one API across miners' models on subnet 103 — const_reborn · 2026-10-30
- SGLang Summit 2026 sells out sponsor booths and Full Passes ahead of Nov 12–13 event — ying11231 · 2026-10-30
- OpenRouter rebuilds provider routing: 46% fewer capacity errors, 16.9% faster TTFT — steipete · 2026-10-30