Qwen3.8-Flash-Next on M3 Ultra: 559 t/s prompt processing, 31 t/s generation
rm-rf-rm · reddit · 2026-09-14
A Reddit user benchmarked Qwen3.8-Flash-Next (Q4KXL GGUF via unsloth) locally on an Apple M3 Ultra using llama.cpp through llama-swap with a 256K context. Measured with llama-benchy: 558.83 t/s prompt processing (pp1000) and 31.05 t/s token generation (tg500, peak 31.67), with 3.3s time-to-first-response. A useful data point for local deployment on Apple Silicon.
More from Infra
- Your AI agent would cost 260% more without prompt caching, Arena data shows — arena · 2026-10-30
- Nebius runs auction-driven GPU pricing, lifting rates ~20% amid tight supply to 2028 — BenBajarin · 2026-10-30
- macOS update silently re-adds eMudhra root certs users had removed — steipete · 2026-10-30
- Bittensor's Terza Inference routes one API across miners' models on subnet 103 — const_reborn · 2026-10-30
- SGLang Summit 2026 sells out sponsor booths and Full Passes ahead of Nov 12–13 event — ying11231 · 2026-10-30
- OpenRouter rebuilds provider routing: 46% fewer capacity errors, 16.9% faster TTFT — steipete · 2026-10-30