Local Qwen runs at 27t/s in chat but drops to 15t/s inside agent harnesses
WizardlyBump17 · reddit · 2026-08-23
A user running Qwen3.6 35B A3B locally via llama.cpp (Intel B580 + 7 5700X3D + 48GB RAM) gets 27 tok/s in normal chat but only 15 tok/s inside coding harnesses like OpenCode and Maki, even with less context. Full Docker launch config (262K ctx, Q80 KV cache, MTP speculative decoding) included; asking why harnesses slow inference down.
More from coding & agent
- Agent evolution: Models absorbing the harness into weights — daniel_mac8 · 2026-08-23
- MCP vs CLI vs API: How Tool Design Drives Your AI Token Bill — sanjaykalra · 2026-08-23
- Distributed network Darkbloom hiring at $100k ARR — gajesh · 2026-08-23
- Optimized SENPAI Prompts, Qwen 3.8 27B Agent Uses Sub-agents More Frequently — morgymcg · 2026-08-23
- Building Great Evals: Avoid Single Scores and Embrace Hill Climbing — realmadhuguru · 2026-08-23
- Stop Extracting Everything: Good Codebases Minimize Jumps — serrjoa · 2026-08-23