M5 Ultra Shootout: Qwen3.8-Flash-Next Prefills 200K in 47s vs Laguna's 455s
nonlinearsystems · reddit · 2026-10-01
A same-day, same-harness benchmark on Mac Studio M5 Ultra 256GB: Qwen3.8-Flash-Next (oMLX, MTP) holds 4,200 tok/s linear prefill, hitting 200K context in 47.1s, while Laguna-S-2.1 degrades superlinearly to 455.4s. Decode: Qwen 59-74 tok/s with 70-76% MTP acceptance; Laguna falls from 68 to 34. Quality tied 4/4 on script-verified problems. Key pitfall: shared prefixes let KV cache leak across runs, faking 21s prefill.
More from Infra
- Mobile AI Factory: agents auto-build on-device SLMs with 19x less GPU time — pmttyji · 2026-11-16
- vLLM serves DeepSeek V4.1-Flash at 954K tokens/s per GPU, $38.96/GPU-hour on GB300 NVL72 — bookwormengr · 2026-11-16
- Colossus 2's 1.2M B200s could generate ~$400B/year in tokens at DeepSeek Flash rates — bookwormengr · 2026-11-16
- BIS issues rule blocking remote access to US AI chips via third-country data centers — teortaxesTex · 2026-11-16
- Qdrant 1.20.1: 50% fewer cold-storage reads, ~2x faster quantization, agent-ready setup — qdrant_engine · 2026-11-16
- One 16GB GPU runs songs, voice, images, 720p video and local RAG — all offline and open source — llama-of-death · 2026-11-16