Benchmark author says OpenRouter unreliably honors Meta Muse effort levels, EU payments broken
PawelHuryn · x · 2026-09-11
Benchmark author Pawel Huryn says he may drop Muse from his benchmark: probes show OpenRouter doesn't reliably support effort levels — token usage shows no correlation with the requested effort. The official API doesn't work for him either: EU Visa/MasterCard payments fail and two weeks of emails got no reply, while buying Kimi or Z.ai credits worked fine. Post ends with an AI PM course promo code.
More from Models
- ToneBench eval: GPT-6.1 Terra ranks #23 at writing — cheaper, not better, than GPT-5.6 Terra — OnlyProggingForFun · 2026-10-28
- Cursor user complains it silently switches to Grok, which fails basic tasks — kylegawley · 2026-10-28
- Musk says Grok Bot keeps improving, calls Vault feature a game changer — beffjezos · 2026-10-28
- Claude refuses to gather legitimately public information, sparking guardrail backlash — mkheck · 2026-10-28
- DesignArena launches Code to Video Arena: models direct 15-45s films purely in code — yshan2u · 2026-10-28
- ChatGPT's 'yes, but with an important clarification' often precedes pure nonsense — Crixusgannicus · 2026-10-28