M1 Pro Benchmarks: Qwen 3.8 Hits 45 tok/s at 256k Context
EyalToledano · x · 2026-09-02
Eyal Toledano shared benchmarks for running Qwen 3.8-Flash-Next on an M1 Pro (64GB RAM). With Q4 quantization and 32k context, it achieves 38 tok/s burst and 51 tok/s aggregate throughput with 4 agents. Even at 256k context using Q3 quantization, it sustains 18 tok/s and reaches 57 tok/s aggregate throughput.
Related event: Qwen3.8-Flash-Next with pMLX Engine Runs Big Models on Small Memory(3 posts)→
More from Infra
- ClickHouse aggregates 100B rows in 0.4s with parallel replicas auto-scaling — Abhishekcur · 2026-10-18
- Auction-based LLM routing paper finds fixed token pricing wastes 10%-71% in costs — _onionesque · 2026-10-18
- Running Ling 3.1 Flash (560B MoE, 25B Active) Across 4 DGX Sparks via llama.cpp RPC — BerryBlush_Rogers691 · 2026-10-18
- SGLang Summit sponsors jump from 11 to 36 in one week — ying11231 · 2026-10-18
- Oracle CDS hits record 300bps, called first AI trade domino after Barclays sell downgrade — arthurcolle · 2026-10-18
- Agent Coding Workflow Would Cost ~$40k/Month on API Pricing, Dev Reveals — ChrisRackauckas · 2026-10-18