New Qwen 3.8 MLX Engine Features Dynamic Quantization
MgkMshrmBrkfst · x · 2026-09-01
The HamsterResearch team is building a custom MLX engine for Qwen 3.8-Flash-Next. Key features include: downloading a single bf16 tiered model that can be quantized on-the-fly to bf16/q8/q4/q3; adjustable n-gram streaming from NVMe; and tunable expert residency ratios. The engine targets 130-150 tok/s aggregate decode speed to support efficient sub-agents.
Related event: Qwen3.8-Flash-Next with pMLX Engine Runs Big Models on Small Memory(3 posts)→
More from Infra
- CoreWeave co-founders cash out $3.3B in stock as AI credit risks mount — SumitGup · 2026-10-18
- ClickHouse aggregates 100B rows in 0.4s with parallel replicas auto-scaling — Abhishekcur · 2026-10-18
- Auction-based LLM routing paper finds fixed token pricing wastes 10%-71% in costs — _onionesque · 2026-10-18
- Running Ling 3.1 Flash (560B MoE, 25B Active) Across 4 DGX Sparks via llama.cpp RPC — BerryBlush_Rogers691 · 2026-10-18
- SGLang Summit sponsors jump from 11 to 36 in one week — ying11231 · 2026-10-18
- Oracle CDS hits record 300bps, called first AI trade domino after Barclays sell downgrade — arthurcolle · 2026-10-18