User Benchmarks Qwen3.8 27b on M5 Max: 8t/s (bf16), 17t/s (8bit)
julianharris · x · 2026-08-16
A user benchmarked the Qwen 3.8 27B model (bf16, 262k context) on an M5 Max with 128GB RAM. Using the oMLX + opencode stack, the speed achieved was 8t/s at bf16 precision and 17t/s at 8bit quantization. The test revealed significant CPU throttling, with expectations that 4bit quantization and speculative decoding will improve performance in the future.
More from Infra
- CoreWeave's debt costs falling: DDTL spreads tightened from S+9.62% to S+2.25% — BenBajarin · 2026-10-16
- SoftBank lines up US IPOs for SB Energy and AI robotics spinoff Roze, valuations top $50B — nordicinst · 2026-10-16
- TRL reproduces Modal's mixed-precision async RL: plain GRPO collapses, SC+MIS holds 0.72 — QGallouedec · 2026-10-16
- Hugging Face scales Indic pretraining with 2.8T tokens and an open data pipeline — Hugging Face Blog · 2026-10-16
- NVIDIA Vera Rubin NVL72 hits up to 5.7x GB300 throughput per MW on agent benchmark — ArtificialAnlys · 2026-10-16
- 14 open-source LLM observability tools to know in 2026, from Langfuse to Laminar — TheTuringPost · 2026-10-16