Together AI's agent eval workflow lifted task completion from 67% to 96%
togethercompute · x · 2026-11-17
Together AI shares its agent eval methodology: define 10 representative tasks, run Claude Code agents in sandboxes, test 3 configs per task (baseline, MCP, skills), then analyze trajectories to improve docs and APIs. Task completion improved from 67% to 96%, with the chart showing baseline at 82%, MCP at 96%, and skills at 97%.
More from coding & agent
- Together AI shares its agent eval playbook, lifting task completion from 67% to 96% — togethercompute · 2026-11-17
- Beaker tunes your agent harness per model before you pick one — VictorSanh · 2026-11-17
- VapiCon 2026 takeaways: phone calls still dominate voice AI, sims and evals go mainstream — chadwallacehart · 2026-11-17
- YC F26 startup OpenScaffold turns past deployments into reusable playbooks for coding agents — ycombinator · 2026-11-17
- Vault-LD: markdown wiki + YAML-LD knowledge graph as an open standard for agent memory — _AustinCalvert_ · 2026-11-17
- An order is not an actor: why we keep reinventing execution models — _AustinCalvert_ · 2026-11-17