Debugging: MTP enabled on Qwen 3.6 9B caused tool calling failures
OvertaxedOne · reddit · 2026-08-29
A developer encountered random MCP and RAG breakage in OpenWebUI when running Qwen 3.5 9B AWQ on vLLM with MTP (Multi-Token Prediction) enabled. After 9 hours of debugging, disabling MTP instantly resolved all tool calling and JSON formatting issues.
- Observation: MTP provided a speed boost but broke downstream tooling on this specific model/setup, while a 3.8 27B model with MTP enabled remained stable.
- Conclusion: Despite the theoretical claim that MTP doesn't change output text, it can cause parsing failures in certain clients or models. Users facing similar tool calling/JSON issues are advised to test with MTP disabled.
More from coding & agent
- Don't Be a Meat Proxy: Process AI Output Before Sharing — bibryam · 2026-08-29
- Dev Critiques OpenSpec: Too Rigid for Green-Field Work — carsonfarmer · 2026-08-29
- MerchantBench Paper Reveals Long-Horizon Agent Performance Collapse — rohanpaul_ai · 2026-08-29
- repofetch: MCP-Powered Agent for Real-Time GitHub Code Search and Q&A — krypticmouse · 2026-08-29
- Motivating Agents to Build Agent-Friendly Tools — doodlestein · 2026-08-29
- Using Frontmatter to Optimize Tabular Data Inputs for LLMs — AttemptTwoThousand · 2026-08-29