Qwen3.5 35B A3B hits 55 tok/s on an RTX 5060 Ti with an extended Garlic build
Azazelionide · reddit · 2026-07-24
A user reports running Qwen3.5 35B A3B float8 at 55 tokens/s on an RTX 5060 Ti using an extended version of Garlic.
They say the result comes from some Gated Delta Network kernel work, and that it significantly outperforms llama.cpp running the same model in Q8 quantization. The reported number drops to 61 tok/s without recording because screen recording consumes CPU/GPU resources.
They also note that this is without MTP, and that MTP could speed generation up further. The author plans to write a blog post explaining the trick behind the speedup.
More from Infra
- RTX 5080 price jumps from $1658 to $2318 in ten days on Amazon — Frizzy-MacDrizzle · 2026-10-27
- ComfyUI on 12GB VRAM: measured peaks show offloading turns 24s into 11 minutes — Impossible-Mail-4619 · 2026-10-27
- Neon rebuilds its docs search entirely on its own stack: Lakebase Search, Functions, AI Gateway — matei_zaharia · 2026-10-27
- PyTorch details 2.12x faster on-device streaming ASR on Arm SME2 via ExecuTorch — PyTorch · 2026-10-27
- NVIDIA shows speculative decoding speeding up LLM inference on Jetson with Qwen3.5 9B — NVIDIA Developer · 2026-10-27
- Revenue per GPU can rise as token prices fall, argues Gavin Baker — GavinSBaker · 2026-10-27