Solo researcher boosts Qwen 3.8 on Mac: +54% decode at 147k context
julianharris · x · 2026-08-29
Independent researcher Youssof released MTPLX V2.10, a full-stack performance overhaul for running Qwen 3.8 on Mac — praised by Julian Harris as a one-man local AI powerhouse, spanning low-level kernel optimizations inside the model plus a custom inference server. Highlights:
- Qwen 3.8 Next support: 4-bit and optimized 4/8-bit models sustain 50–80 TPS at high contexts
- mmap streaming: the 32 GB n-gram table streams from SSD via mmap instead of RAM; both models fit a 96 GB Mac with zero speed impact
- 27B decode & prefill boost everywhere: +15% TPS at 3k context, +29% at 88k, +54% at 147k; prefill +41%
- 48 GB Macs: from 3–4 TPS in swap at 30k context up to 33 TPS
- KV cache solved: 8-bit and 4-bit caches now barely hurt speed, versus a prior 50% decline
Harris suggests Jeremy Howard's Answer.ai should snap him up before Big Tech does.
More from Infra
- AI's smartest bear case is supply: $500B GPUs idle, but demand is elastic — bittingthembits · 2026-10-15
- Intel said to have secured foundry commitments from Broadcom and Google, Apple wins on 14A — BenBajarin · 2026-10-15
- Magnitude open-source inference engine beats Ollama by over 2x with auto-tuned kernels — nickbaumann_ · 2026-10-15
- Nvidia to announce $28B in AI investments in Europe this week, report says — georgemillo · 2026-10-15
- A $10,811 Cloudflare Bill From an Unused AI-Written Test Project: Refunds Aren't a Safety Net — sujingshen · 2026-10-15
- Retraining-Free Method Turns Transformers Into Elastic Models Adapting to Any Compute Budget — VISLab-Amsterdam · 2026-10-15