NVIDIA SkillSpector on 4 popular Agent Skills repos: read the findings, not the score
viest_me · reddit · 2026-11-16
The author ran NVIDIA SkillSpector v2.12.0 in static-only mode (--no-llm) inside a throwaway container against four trending Agent Skills repos: anthropics/skills (20 skills, 250 findings, 112 HIGH), addyosmani/agent-skills (25 skills, 59 findings, 32 HIGH), mattpocock/skills (39 skills, 18 findings, 9 HIGH), and google/skills (only 27 of 203 skills scanned; the 176-skill cloud folder returned an empty report after 267s).
Findings fell into three groups:
- Coverage gaps counted as HIGH: the dominant analysis-evasion category (71 of anthropics' 250 findings) mostly fires when the scanner failed to finish reading a referenced file — a scan gap, not evidence of hiding anything, but it inflates scores.
- Defensive text read as attacks: instructions telling the agent to ignore injected page text got flagged HIGH "Instruction Override"; a security-hardening skill scored 100 partly from a YARA match on its "Threat Model First" section. Static scanners can't distinguish quoting an attack from performing one.
- Real capabilities correctly flagged: mattpocock's git-guardrails writes to /.claude/settings.json (accurate "Agent Config Directory Access"), and Google's google-ads-api-quickstart walks through clientsecrets.json and gcloud credential files (accurate "Credential Access").
Takeaways: read findings rather than scores; prioritize flags about agent config, credential files, and bundled scripts; scan hook scripts that actually execute, not just skill folders; a single static run may produce noise the LLM stage could filter (untested).
More from coding & agent
- A satirical year in AI coding: nobody writes code anymore, and we're still figuring out what that means — mattpocockuk · 2026-11-16
- Executor overhauls agent approvals with 4 modes, adds Gmail-style undo for any agent action — threepointone · 2026-11-16
- Cheap-first routing: only 4 of 143 questions escalated to a frontier model, cost down 80% — xrplincoln · 2026-11-16
- I Shipped an App, a Site, and a CRM With Codex and Claude Code — and Learned Nothing — ThatGuyJord2020 · 2026-11-16
- Over Half of LLM Requests Go to Models Younger Than 60 Days, Vercel Gateway Data Shows — aruthesa · 2026-11-16
- Open-source Craft Anything hits 34K GitHub stars in 16 days by letting Claude Code clone any app — thisguyknowsai · 2026-11-16