Anthropic's claude-api skill adds build-eval and hillclimb commands for automated evals

brada · x · 2026-09-29

A claude.dev blog post by Lance Martin details the eval design and hillclimbing guidance now built into the claude-api skill. Two commands: /claude-api build-eval creates an evaluation inside your codebase; /claude-api hillclimb improves the app against it one change at a time, with held-out examples to catch overfitting. Key principles: eval tasks should mirror production; stronger models and more thinking should score better (otherwise tasks are ambiguous or the grader miscalibrated); and the frontier model at max effort should land well below 100% so changes can be reliably judged.

Related event: Anthropic Adds Automated Eval Building and Hillclimbing Commands to Claude Code(6 posts)→

Original post →

More from coding & agent

coding & agent channel →