antirez: DwarfStar runs Qwen3.8 Flash Next on 64GB Macs at 50-70 tok/s with SSD n-grams
antirez · x · 2026-09-14
antirez announced that DwarfStar now supports Qwen3.8 Flash Next, with the work done by @ivanfioravanti in a PR.
- Performance: runs well on 64GB Mac systems with 50-70 tok/s inference and >1400 tok/s prefill; Metal-only for now.
- Technique: uses the same SSD-backed n-grams approach as DS4.1F to offload n-gram caches to disk.
In a follow-up he notes the setup may or may not make sense on the 128GB DGX Spark (extra speed plus free RAM are tempting), but is clearly worthwhile for ROCm users with 64GB Framework Desktops.
More from Infra
- Musk: AI data center contracts to save Georgia customers nearly $1B, ~$180/year on bills — elonmusk · 2026-09-14
- Meta model reached external systems during cybersecurity test due to sandbox misconfiguration — emmanuelvivier · 2026-09-14
- Scientists Create Functional Viruses Using AI, Evading DNA Screening — emmanuelvivier · 2026-09-14
- UN warns AI data centers are outpacing power grid buildout by years — emmanuelvivier · 2026-09-14
- Y Combinator Demo Day spotlights floating nuclear data centers and inference chips — emmanuelvivier · 2026-09-14
- K2 Horizon Lineup Hits Artificial Analysis, but Awful KV Cache Design Misleads Its Charts — crusaderky · 2026-09-14