Openbenchmarks ranks web search APIs for coding agents; Tiny_Fish takes #1 on cost with 46% fewer tokens
Scobleizer · x · 2026-09-10
Openbenchmarks released the "Web Search for Coding Agents" benchmark (2026): 100 realistic product tickets where the agent must patch inherited code by retrieving the vendor's official docs—key implementation details are never revealed in the ticket, patches must compile, and cited URLs must genuinely appear in that run's search results, blocking memorized answers.
Key insight: coding is the #1 AI agent use case, but most token budget goes to searching for API docs, changelogs, and best practices—not writing code. Search is where cost and quality collide: cutting tokens hurts accuracy, protecting accuracy balloons the bill.
TinyFish climbed from 7th to 3rd on search-and-fetch, ranking #1 on token efficiency and cost per task, overtaking Perplexity and Parallel Web Systems and closing in on Exa. After a revamp two weeks ago, task completion rose from 69% to 79%—just 4 points behind leader Exa Deep—while using 46% fewer tokens and costing 60% less per task.
More from coding & agent
- Replace RPA by recording your actions into Doubao Skills: a hands-on walkthrough — lxfater · 2026-09-10
- DeepSeek's new open model beats GLM 5.3 and Kimi K3 at 4-10x lower price — deedydas · 2026-09-10
- User drops GPT-6 Astra after a bad week: weak at orchestration, back to GPT-5.6 Sol — koltregaskes · 2026-09-10
- FitGate: open-source tool makes Claude Code do a micro-workout before approving permissions — StudyLabsIndia · 2026-09-10
- Smash Bros Melee fully decompiled over 6 years, now runs in MR on Quest — NathanpmYoung · 2026-09-10
- The One Thing That Separates a Chatbot From an Agent: Function Calling, Explained — Al_Grigor · 2026-09-10