Measuring Agent Skills With Evals: How to Tell if an Instruction File Actually Helps
blaizedsouza · x · 2026-10-20
An article by sermakarevich targets engineers who run coding agents and must decide which instruction files (skills) to install or write.
- The problem: millions of skills exist, hardly any have ever been tested, and most were vibe-written — the whole skills industry is subjective.
- The method: take any skill, extract the ideas it rests on, and measure in two numbers whether it actually helps.
A reusable approach for teams drowning in skill lists and wanting evidence before installing.
More from coding & agent
- Claude Code v2.1.304 adds guided auto-mode setup, stricter permissions and usage warnings — ashwin-ant · 2026-10-20
- Pixel: a library bringing 120 FPS graphical UIs with Chromium WebView to the terminal — aidenybai · 2026-10-20
- Swarms argues Collective Superintelligence (CSI) will surpass AGI and ASI — KyeGomezB · 2026-10-20
- AI is already doing real work in RF chip design, FPGA optimization and 3D CAD, engineers say — mark_k · 2026-10-20
- Code review isn't dead: half of large AI-generated diffs add requirements you don't have — rseroter · 2026-10-20
- Rule of thumb: spend 90% of your RL task compute on trying to break them — marktenenholtz · 2026-10-20