Qwen 3.6 35B MoE runs on a Xiaomi 12 Pro with 12GB RAM at 2.4 tok/s
Aromatic_Ad_7557 · reddit · 2026-07-24
A Reddit user reports running Qwen 3.6 35B MoE in Q4KM on a modified Xiaomi 12 Pro with just 12GB of RAM.
Using BigMoeOnEdge, the setup reaches 2.428 tok/s, with TTFT around 19.9s, 8k context, and heavy CPU usage plus paging. The post includes detailed token/compute/cache stats and shows that a very large MoE model can run locally on a phone-class edge device, albeit with clear memory and latency trade-offs.
More from Infra
- Cloud capacity crisis is now an AI architecture problem, argues new newsletter — DavidLinthicum · 2026-10-26
- SpaceX files $40B GPU financing commitment with Apollo, already at 1.7GW — XFreeze · 2026-10-26
- Meta plans to raise up to $95B to fund massive AI expansion — Polymarket · 2026-10-26
- Should the community archive open model weights in case Hugging Face gets restricted? — Tall-Ad-7742 · 2026-10-26
- What happens to neocloud GPUs when the AI datacenter debt bubble bursts? — bowdoin-yale · 2026-10-26
- Nebius CEO: Debt Markets Closed for Neoclouds Last Week, Leaning on Customer Advances — zephyr_z9 · 2026-10-26