Researchers Call for Better Benchmarks as Model Routing Faces Criticism

Amid criticism of model routing services like OpenRouter, researcher Elvis Saravia argues most critiques lack evidence and calls for better benchmarks, noting his own agent routing tests were promising but may not generalize beyond specific tasks.

2026-09-27 ~ 2026-09-27 · 2 related posts

• omarsar0 on custom agent benchmarks: public numbers don't mean much, Jev Router rivals GPT-6 Astra low — omarsar0 · 2026-09-27
• Routing Evals Are Missing, Says Elvis Saravia as OpenRouter Criticism Rages Without Evidence — omarsar0 · 2026-09-27