Local Qwen runs at 27t/s in chat but drops to 15t/s inside agent harnesses
WizardlyBump17 · reddit · 2026-08-23
A user running Qwen3.6 35B A3B locally via llama.cpp (Intel B580 + 7 5700X3D + 48GB RAM) gets 27 tok/s in normal chat but only 15 tok/s inside coding harnesses like OpenCode and Maki, even with less context. Full Docker launch config (262K ctx, Q80 KV cache, MTP speculative decoding) included; asking why harnesses slow inference down.
More from coding & agent
- Vercel Goes Full Backend: Python GA, Containers, and Daemons — cramforce · 2026-10-16
- WorldMemArena Accepted at NeurIPS: Memory Storage ≠ Memory Use — xwang_lk · 2026-10-16
- Vercel Ship Demos Container Deploys for Go, Ruby, Java and More — edwin · 2026-10-16
- TRL adds day-0 support for fine-tuning Muse Spark Open's 816B MoE with LoRA on 64 H100s — QGallouedec · 2026-10-16
- Databricks engineers land impactful PRs on day 2 with AI dev tools, vs week 3 before — Yuchenj_UW · 2026-10-16
- Perplexity open-sources SPACE, the sandbox platform powering its Computer agents — AravSrinivas · 2026-10-16