safeagents judge bench lands on HF: official leaderboard for decision-model safety judges
yb2698 · x · 2026-11-10
safeagents judge bench now has an official leaderboard on Hugging Face: the dataset contains 661 real agent tool calls flagged by the scanner, each labeled allow/block plus a misuse type (reward hacking, harness introspection, credential misuse, etc.), Apache-2.0 licensed.
The leaderboard scores small decision models as agent safety judges by balanced accuracy with parameter-size filters; some current scores: d3-ultra 0.76, d3-turbo 0.72.
Related event: safeagents judge bench v2 launches leaderboard on Hugging Face(3 posts)→
More from coding & agent
- YC F26's 0.security Launches Autonomous Security Team That Ships Fixes, Claims 30+ Linux 0-Days — ycombinator · 2026-11-11
- DSPy 3.5 released: framework is now 85% smaller and production-ready — dbreunig · 2026-11-11
- A Swarm of 10 DeepSeek Agents With an Intercom Turns Out Extremely Effective at Finding Exploits — andersonbcdefg · 2026-11-11
- DIY AI neon sign: LILYGO display + Hermes Agent + LFM3-Audio voice terminal — alexcovo_eth · 2026-11-11
- Vercel Security Review flags 57,897 vulnerabilities in first week, agent auto-patches 14,665 — cramforce · 2026-11-11
- RLVR lets small models close the coding gap; Python's dynamic typing hurts pretraining — mdancho84 · 2026-11-11