Together AI shares its agent eval playbook, lifting task completion from 67% to 96%
togethercompute · x · 2026-11-17
Together AI explains how it systematically evaluates how well coding agents can use its platform, arguing that developer experience (DX) is becoming agent experience (AX): the easier your software is for agents, the easier it is for devs to build with.
Their process:
- Define 10 representative tasks
- Run agents (Claude Code) in sandboxes
- Test 3 configs per task (baseline, MCP, skills)
- Analyze trajectories and improve docs and API errors
Small fixes to docs, skills, and error messages lifted task completion from 75% to 97%. They also cite GitHub data: agents opened about 40% of all PRs in 2026, with platform activity tripling year over year. A rare public, hands-on example of agent eval engineering from a vendor.
More from coding & agent
- EMBRACE: Embedded Browsers in Cursor, Copilot, Codex, VS Code, Claude Desktop All Hacked — cyb3rops · 2026-11-17
- OpenCode Browse 0.1.0 Rewritten in Rust: 25x Less Memory, Playwright Dropped — gaganghotra_ · 2026-11-17
- Together AI's agent eval workflow lifted task completion from 67% to 96% — togethercompute · 2026-11-17
- Codex and Claude MCP Wired into DaVinci Resolve Hailed as a Standout Workflow — _AustinCalvert_ · 2026-11-17
- Beaker tunes your agent harness per model before you pick one — VictorSanh · 2026-11-17
- VapiCon 2026 takeaways: phone calls still dominate voice AI, sims and evals go mainstream — chadwallacehart · 2026-11-17