Local Reasoning Configs: Managing Qwen's Thought Budget
Altruistic_Heat_9531 · reddit · 2026-08-18
The author shares llama.cpp configuration tips for running local models (specifically Qwen 3.8), focusing on controlling "reasoning effort" to balance performance and resource usage.
Key Strategies:
- Native Support: Models like Qwen 3.8 and DeepSeek V4 natively support effort levels (low, medium, high, xhigh).
- Token Budgeting: For models lacking native support (e.g., Qwen 3.5/3.6), a token budget is used to cap reasoning length.
- Hybrid Limits: Even with native levels, a hard max reasoning token cap is applied.
- Non-Preserving Mode: An xhigh-no-preserve variant uses reasoning as a scratchpad without saving it to history, saving context window space.
The author notes that low effort is sufficient for most use cases.
More from coding & agent
- Meta RPM Model Prioritizes AI Research Experiments, Cuts Compute by 40% — rohanpaul_ai · 2026-08-18
- Talking to Codex: Why conversing with the computer helps clarify ideas — gabrielchua · 2026-08-18
- How should a LangGraph supervisor route multiple agents with interrupts in one session? — keep__it_simple · 2026-08-18
- Slice.js Documentation MCP Released: Enables Full-Text Search and Retrieval — modelcontextprotocol · 2026-08-18
- SearchShop AI: AI Commerce for Shopify via MCP Connector — modelcontextprotocol · 2026-08-18
- fireworks-tech-graph Skill Hits 10K Stars in Four Months — vista8 · 2026-08-18