Addy Osmani shows Claude Code can auto-evaluate whether a plugin improves answers
addyosmani · x · 2026-09-21
Addy Osmani reminds devs that Claude Code can actually test whether a skill or plugin improves Claude's answers:
- claude plugin eval init — describe what a good result looks like and Claude writes the test cases into your plugin folder;
- run claude plugin eval . to execute them.
This gives plugin/skill authors a repeatable eval loop without hand-writing test harnesses, making it possible to measure real answer-quality gains.
More from coding & agent
- Coding agent UX gripe: Astra says "PR is up" without linking to it — altryne · 2026-09-21
- God's Eye View: open-source spy-satellite simulator with real data hits 39.5k GitHub stars — alex_verem · 2026-09-21
- GEPA Prompt Optimization Lifts Jev's F1 From 69.1% to 79.7% on Medical Literature Task — matei_zaharia · 2026-09-21
- Superlinear Episode Details the Fall 2026 Workflow for Starting Projects with Coding Agents — samgoodwin89 · 2026-09-21
- /brag: Open-Source Claude Code Skill Turns Your Project Into a Launch Video With One Command — tom_doerr · 2026-09-21
- Big Tech New Hire Says Everyone Ships Claude Code Output 13 Hours a Day Without Reviewing It — MrMenuk · 2026-09-21