Agent sandbox test: auto-approve let exfiltration to new hosts through 12 of 12 times
No-Peanut-6988 · reddit · 2026-10-23
Sorami tested what NVIDIA's open-source agent sandbox OpenShell 0.1.2 actually blocks, using a local agent (qwen3:8b via Ollama) on a project containing a fake secret-exfiltration script:
- No sandbox: secret escaped 10/10 runs.
- OpenShell default policy: 0/10, every exfiltration path blocked, no bypass of documented controls found.
But once convenience options were enabled:
- Auto-approve: new public hosts approved without a human 12/12 times, including rules the sandbox drafted itself from blocked connections.
- "GET only" rules still leaked via query strings and headers.
- Audit mode (default for new rules) logs traffic and lets it through.
- GraphQL/MCP/WebSocket/JSON-RPC rules were flagged unsupported but the loader accepted them anyway.
None of this is a bug—it's documented—but the sandbox is only as strict as whoever approves new access, and auto-approve meant nobody did. Limits: one version, one laptop, a weak model.
More from coding & agent
- MIT Prof Built an AI Playbook That Audits Academic Websites and Simulates Visitors — soumitrashukla9 · 2026-10-23
- Robot arm Podrick learns to load utensils by distilling lessons from its own failed runs — ycombinator · 2026-10-23
- Agents that don't close connections on retry waste ~148 tokens per task, vLLM tests show — No-Peanut-6988 · 2026-10-23
- For writing JSON schemas, Claude is the best pick and Gemini the worst, dev says — gaganghotra_ · 2026-10-23
- OSI Hackathon winner Consilith: an open multiplayer studio where researchers and their AI agents collaborate on science — BenBlaiszik · 2026-10-23
- Claims Engine: a graph-based system connecting evidence to claims — anselm · 2026-10-23