skills-evals: CI Regression Testing for AI Skills
Key_Intention_2671 · reddit · 2026-07-16
The author found their instruction/skill files becoming unstable as the codebase grew and APIs changed, so they built a tool called skills-evals.
It's a zero-dependency tool that validates, triggers tests, and runs regression tests on AI-related artifacts in CI, covering:
- skills.md
- Copilot instructions
- Claude custom agents
- Cursor rules
Core function: if a code change causes a skill to stop triggering as expected or change behavior, CI fails. It's also compatible with Anthropic's evals.json schema.
More from coding & agent
- Soft Clamp cuts tool-call overuse in multi-teacher distillation, from 13.7% to 9.0% — antgroup · 2026-07-21
- Agent harness memory loss and compaction are still a major usability problem — adityaag · 2026-07-21
- SpecJudge runs locally on Ollama to pick the right-sized AI model for your project — jokiruiz · 2026-07-21
- A developer maps out six design rules for CLIs that humans and AI agents can both use — yujiezha · 2026-07-21
- A coding-agent skill that forces ADHD-friendly, answer-first output — ayghri · 2026-07-21
- A set of agent skills for CAD, robotics, and hardware design — earthtojake · 2026-07-21