Anthropic adds official eval-building and hillclimbing workflow to Claude Code skill
ClaudeDevs · x · 2026-09-29
Anthropic (claude.dev blog, by Lance Martin) added two commands to the claude-api skill so Claude Code can build evaluations and improve apps against them automatically:
- /claude-api build-eval: set up an evaluation inside your codebase
- /claude-api hillclimb: improve the app one change at a time, with a held-out example set to catch overfitting
Key eval design principles from the article:
- Eval tasks must mirror production — not be chosen because they're easy to generate or grade
- Stronger models should score higher; if not, tasks are ambiguous or the grader is miscalibrated
- Keep passable headroom at the frontier — the best model at max effort should sit well below 100%, or you can't reliably judge the impact of changes
An official, hands-on guide to eval engineering and agent iteration workflows.
More from coding & agent
- Agents Now Reply to Slack Messages on Their Own, Resolving a Prod Incident Ghost-to-Ghost — iamrobotbear · 2026-11-16
- Engineer's Interview Prep Method: Use Claude to Find Which Answers Undersell Your AI Work — brandon_galang · 2026-11-16
- Agent asks permission before clicking "I'm not a robot" checkbox — Puzzleheaded-King584 · 2026-11-16
- Generic API-catalog MCP server vs one tool per job: a context-bloat experiment — MandeepKaur9407 · 2026-11-16
- Berkeley paper: LLMs answer with stale values even when the update is in context — ombre-ai · 2026-11-16
- Effective HTML: 4.2k-star agent skills turn wireframes into self-contained interactive prototypes — tom_doerr · 2026-11-16