skillEd: open-source regression testing loop for policy-driven agent skills
MiserableFlatworm337 · reddit · 2026-10-23
The author released skillEd, an Apache-2.0 evaluation toolkit targeting agent skills where a plausible answer still violates written policy. The repo's refund skill example refunded shipping, rounded the wrong way, and applied a 14-day cutoff instead of 30—fixing the wording is only half the job; the same cases must rerun to catch new regressions.
The workflow keeps YAML cases and fixtures alongside SKILL.md, validates them, saves a baseline, then compares revised skills on unchanged cases plus separate holdouts. Exact assertions check amounts; model judges handle qualitative checks. Reports come in JSON, HTML, and JUnit, with CLI and MCP routes so a coding agent can run the loop itself. The repo currently has 15 stars.
Related event: Open-source skillEd brings regression testing to agent skills(2 posts)→
More from coding & agent
- Agents that don't close connections on retry waste ~148 tokens per task, vLLM tests show — No-Peanut-6988 · 2026-10-23
- Agent sandbox test: auto-approve let exfiltration to new hosts through 12 of 12 times — No-Peanut-6988 · 2026-10-23
- For writing JSON schemas, Claude is the best pick and Gemini the worst, dev says — gaganghotra_ · 2026-10-23
- OSI Hackathon winner Consilith: an open multiplayer studio where researchers and their AI agents collaborate on science — BenBlaiszik · 2026-10-23
- Claims Engine: a graph-based system connecting evidence to claims — anselm · 2026-10-23
- One Claude Code session ran autonomously for 53 hours via orchestrators orchestrating orchestrators — wonderclown17 · 2026-10-23