Not Diamond proposes new benchmarks for evaluating model routing in AI agents
rohanpaul_ai · x · 2026-09-02
Not Diamond released a document detailing their benchmarking methodology for evaluating model routing in AI agents. They argue that traditional approaches like single-turn and first-turn routing are ineffective and brittle for long-horizon agent workloads due to KV cache invalidation and inability to handle mid-session complexity shifts. Not Diamond Code uses a novel routing technique that predicts future rewards and costs at each step in a cache-aware manner, taking into account session state, token counts, and task complexity. The company also shares tooling for clients to benchmark their routing solutions independently.
Related event: Not Diamond Unveils Agent Model Routing Method, Cutting Costs Up to 80%(2 posts)→
More from coding & agent
- Anthropic borrows lawyers' ethics for Claude agents that closed 186 deals with no human approval — rohanpaul_ai · 2026-10-18
- Prompt injection via email can hijack your AI agent — jev-guard adds risk-scored guards — sujingshen · 2026-10-18
- 14-Agent AI Film Pipeline on One RTX 5090: Fixing Cut Defects Between Clips — developervkmp · 2026-10-18
- Meta paper introduces "agent plasticity": best-performing agent isn't the most efficient learner — rohanpaul_ai · 2026-10-18
- Dev ships a bushfire prediction platform in a weekend with frontier models — CtrlAltDwayne · 2026-10-18
- 10 open-source GitHub repos that let AI agents control your phone, from Open-AutoGLM to mobile-use — victor_explore · 2026-10-18