Higher reasoning effort doesn't help AI agents catch silent failures, τ-bench test finds
TruthDeep8659 · reddit · 2026-10-31
While testing agents on τ-bench, the author found LiteLLM's dropparams=True silently dropped reasoningeffort, so 'high' runs weren't high effort at all. After fixing it via OpenRouter's reasoning.effort field: high effort caught 11 of 59 silent save failures vs 10 at low (McNemar p=1.00), while reasoning volume grew 2× for Claude Sonnet 5.5 and 43× for GPT-6.1 Sol. Every catch came from a later save's response, never from the agent re-checking. Code and write-up linked.
More from coding & agent
- Three.js merges Nanite-style meshlet pipeline, rendering 1B triangles at 150FPS — anselm · 2026-10-31
- Steve Yegge: Beads hits 4 transactions/sec worldwide, 500k downloads this month — Steve_Yegge · 2026-10-31
- agent-stat 0.3.0 tracks your Codex and Claude Code usage and costs locally — talkaboutdesign · 2026-10-31
- Northflank hacked, all env vars stolen; key-hoarding platforms are supply-chain magnets — andersonbcdefg · 2026-10-31
- Game dev shares 'character gym' workflow: ask AI to make every movement parameter tunable — chongdashu · 2026-10-31
- Google XR Labs lead demos on-device spatial AI with XR Blocks, LiteRT.js and Gemini — DuRuofei · 2026-10-31