OpenRouter Silently Drops Reasoning Effort Params, Skewing Model Benchmarks
PawelHuryn · x · 2026-08-07
Developer Pawel Huryn discovered inconsistent scores (14 vs 17 out of 105 bugs) when testing Meta's Muse Spark 1.2 through different API routes.
Investigation revealed that OpenRouter silently ignores the reasoningeffort parameter. While sending an invalid value to official APIs (like Meta's or xAI's) correctly returns a 400 error, OpenRouter returns a 200 OK but applies nothing. Consequently, setting effort to high for models like Kimi K3 and DeepSeek on OpenRouter actually defaults to standard effort.
The author also shared an updated model routing table based on these findings: Fable for judgment, Sol for heavy coding, Grok for daily task subagents, and Luna + max effort for economic runs.
Related event: OpenRouter Caught Dropping Inference Parameters, Distorting Model Scores(2 posts)→
More from coding & agent
- Useful Hermes Prompt Tip: Make Agents Verify Code Changes — alexcovo_eth · 2026-08-07
- Developer Releases Interactive and Browser Control Plugins for Prime Agent — daniel_mac8 · 2026-08-07
- Exploring Proactive AI: Auto-Spawning Sub-Agents Based on Chat Context — thisiskp_ · 2026-08-07
- Malwoverview: Open-Source Malware Hunting Tool with LLM-Powered IOC Extraction — tom_doerr · 2026-08-07
- Developer Gives Claude $90 and a Domain, AI Builds Autonomous On-Chain Q&A Product — No_Departure_9908 · 2026-08-07
- Give Cloud AI Agents a Real Mac with Xcode in 60 Seconds — smtabatabaie · 2026-08-07