Qwen3.6 35B MoE hits 19.5 tok/s on a 16GB 4060 Ti, 10x faster than dense models
dreamyrhodes · reddit · 2026-10-17
A Reddit user ran Qwen3.6 35B A3B (Q5KXL, 26GB, 132k context, q80 KV cache) via llama-server on a 4060 Ti 16GB with an old i7 and 48GB DDR4, reaching 19.5 tok/s generation and 132 tok/s prompt eval — about 10x the 2 tok/s of similarly sized dense models, with only 4.5GB VRAM used via --cpu-moe. The thread explores why streaming expert layers over PCIe from DDR4 isn't the bottleneck: MoE activates few experts per token, so little data moves.
More from Infra
- Ethereum Foundation's zkAPI brings anonymous frontier AI access via zero-knowledge proofs — DavideCrapis · 2026-10-17
- Goldman: $738B memory stock wipeout after CXMT 4F² reveal was overreaction — zephyr_z9 · 2026-10-17
- Ben Horowitz: a16z avoids neoclouds — bandwidth went from $1,500 to $50/Mbps and wiped out holders — SumitGup · 2026-10-17
- TokenSpeed ships Day-0 DeepSeek V4.1 support, fastest engine on Hopper GPUs — zhyncs42 · 2026-10-17
- ChinaTalk: Saif Khan argues DUV controls could leave China with 100x fewer AI chips by mid-2030s — ChinaTalk · 2026-10-17
- HN Debate: Is MCP Becoming the Common Foundation for Smart, Decentralised Search? — contextetech · 2026-10-17