Anthropic Adds Automated Eval Building and Hillclimbing Commands to Claude Code
On 09-29 Anthropic published an official guide by Lance Martin on the claude.dev blog, adding two commands to the claude-api skill that let Claude Code automatically build evals inside a codebase and iteratively optimize applications based on them; the skill has been open-sourced for community reuse.
Confirmed
- build-eval: /claude-api build-eval runs directly in the codebase and supports auto-building evals from production traces.
- hillclimb: iteratively hillclimbs the application against the built evals. Per Arindam1729, the combo raises accuracy while cutting cost; AGI Hunt cites the official example where customer-support benchmark accuracy rose from 78.6% to 90.5% at one-fifth the cost.
- On 09-30 Lance Martin shared the repo location: skills/claude-api/shared/eval under the anthropics/skills repository, with cost and eval-hillclimbing guidance.
- Lance Martin responded to feedback from Hamel Husain, saying the skill will be updated accordingly.
Why it matters
- Building eval sets used to require heavy manual effort; generating evals automatically from production traces lowers the barrier dramatically, closing the loop of build–eval–optimize in Claude Code application development.
- The skill is open source and iterates on community feedback, so third-party developers can reuse or improve the workflow; Lance Martin's post also lays out methodology for designing evals and hillclimbing without fooling yourself.
2026-09-29 ~ 2026-09-30 · 6 related posts
Primary sources
- Anthropic adds build-eval and hillclimb commands to automate eval design with Claude — RLanceMartin ·
- Anthropic open-sources claude-api eval skills with cost and eval hillclimb guides — RLanceMartin ·
- Anthropic open-sources its eval design and hillclimbing playbook: 78.6%→90.5% accuracy at 1/5 cost — AGI Hunt ·
- Anthropic adds official eval-building and hillclimbing workflow to Claude Code skill — ClaudeDevs · 2026-09-29
- [source] Anthropic adds build-eval and hillclimb commands to automate eval design with Claude — RLanceMartin · 2026-09-29
- build-eval + hillclimb bring real eval tooling to Claude Code agents — Arindam_1729 · 2026-09-29
- [source] Anthropic open-sources claude-api eval skills with cost and eval hillclimb guides — RLanceMartin · 2026-09-30
- [source] Anthropic open-sources its eval design and hillclimbing playbook: 78.6%→90.5% accuracy at 1/5 cost — AGI Hunt · 2026-09-30
1 near-duplicate retellings: brada