You Can't Tell What an Agent Skill Will Do by Reading It: Zenity's Detonation Chamber
AI Engineer · youtube · 2026-11-11
Roey Ben Chaim of Zenity (MATS Fellow) shows why reading SKILL.md cannot establish what an agent skill will do: an official diagnostic skill can read cloud credentials, contact identity services and probe metadata endpoints — actions indistinguishable from an info stealer. Instructions fetch more instructions, install scripts pull packages, and package-created processes may never appear in the agent's own trace.
Approach: an agent detonation chamber — sandbox, a world builder supplying expected services, an operator exercising paths, and instrumentation for processes, filesystem and network traffic.
Cases: chained skill installs, misleading package names, persistence, hook abuse, remotely fetched prompts; one investigation traces a lookalike project through a downloaded payload to credential harvesting — the full chain matters more than the entry file.
Takeaway: treat skills as supply-chain dependencies — inspect what they load, restrict access, preserve execution evidence beyond the model's visible conversation.
More from coding & agent
- ZenML Pro launches always-on AI engineering team to keep agents in production — strickvl · 2026-11-11
- Latent Space tests NaiveAI's 2,100 tok/s coding model: fast, verbose, promising — Latent Space · 2026-11-11
- Anaconda's LLMOps guide: retrieval quality degrades independently of the model — anacondainc · 2026-11-11
- Dev demos agent with 'durable object brain' plus on-demand computer, local or remote — samgoodwin89 · 2026-11-11
- Right-sizing intelligence per role: frontier reasoning at the control plane, adversarial verifiers at the gate — clmt · 2026-11-11
- MCP Apps ship with your server and run in any AI app supporting the extension — ClaudeDevs · 2026-11-11