ShellRisk-Bench: A Benchmark for Evaluating Cyber Risk of Agent Tool Calls
lhoestq · x · 2026-08-21
ShellRisk-Bench is a new benchmark designed to evaluate the cyber risk of agent tool calls.
- Data Source: Sourced from real agent trajectories and adversarial corpora, unlike other benchmarks.
- Goal: Built to test guardrails at the base rates seen in actual production traffic.
More from Safety
- Anthropic changes data retention policy after enterprise pushback — The Decoder · 2026-08-21
- UK's AI Security Institute names Henry de Zoete as new Director — HZoete · 2026-08-21
- Discussion: Data Privacy in AI Agents and the Case for Private Inference — Many_Audience7660 · 2026-08-21
- AI agent finds critical vuln, earns $1M bounty on Box cloud VMs — Scobleizer · 2026-08-21
- Collective AI Swarm Might Precede ASI: BlackHC on Multi-Agent Intelligence — BlackHC · 2026-08-21
- Does Copyright Protect AI-Generated Content in Europe? — EUobs · 2026-08-21