Qwen 3.8 Flash Next tested on a 128GB Mac: smarter and faster than 27B, needs ~80GB RAM
julianharris · x · 2026-09-01
Julian Harris tested Qwen 3.8 on a MacBook Pro 14" M5 Max / 128GB using MTPLX, a vertical local-LLM stack for Macs:
- 27B throughput drops significantly as the context window grows;
- Flash Next ("Qwen 4 preview", MoE 125-a6b + 4-bit + ngram caching) degrades far less — not as good as cloud Opus, but smarter, faster and more efficient overall;
- That config needs 80GB RAM, so it's really only viable on 128GB Macs; flash-next does clever offloading but performance 'sinks like a stone' AFAICT.
More details coming on his blog AI Local.
Related event: Qwen 3.8 Tested on 128GB Mac: Smarter but Memory-Hungry(2 posts)→
More from Infra
- ClickHouse aggregates 100B rows in 0.4s with parallel replicas auto-scaling — Abhishekcur · 2026-10-18
- Auction-based LLM routing paper finds fixed token pricing wastes 10%-71% in costs — _onionesque · 2026-10-18
- Running Ling 3.1 Flash (560B MoE, 25B Active) Across 4 DGX Sparks via llama.cpp RPC — BerryBlush_Rogers691 · 2026-10-18
- SGLang Summit sponsors jump from 11 to 36 in one week — ying11231 · 2026-10-18
- Oracle CDS hits record 300bps, called first AI trade domino after Barclays sell downgrade — arthurcolle · 2026-10-18
- Agent Coding Workflow Would Cost ~$40k/Month on API Pricing, Dev Reveals — ChrisRackauckas · 2026-10-18