1,000-Test Eval of LLMs as Agent Safety Gates: First Instinct Beats Deep Thinking
Xianbao_QIAN · x · 2026-08-09
To prevent LLMs from accidentally executing dangerous commands while avoiding excessive interruptions, the author created 1,000 test cases to evaluate several mainstream LLMs acting as "safety gates" for agents.
Key Findings:
- No clear winner: The best models let through only 2% of weighted danger but interrupt 70% of normal operations.
- Open vs. Closed: Open-weight frontier models surprisingly outperformed closed models in both minimizing interruptions and catching dangers.
- Instinct over deep thinking: Taking Kimi K3 as an example, with reasoning mode enabled, it misses 8.6% of dangers. When answering with a quick YES/NO instinct, the miss rate drops to 2.5%.
- Strong Base64 decoding: With thinking mode on, modern models can easily catch and decode hidden dangers masked in Base64 strings.
More from coding & agent
- Paper: Comparative Approaches to Agent Retrieval over Large Skill Libraries — alex_verem · 2026-08-10
- Edge Caching for AI Agents: Return High-Frequency Requests Directly at the Edge — blaizedsouza · 2026-08-10
- Osintgraph: Open-Source AI Tool for Instagram Social Network Graphing — tom_doerr · 2026-08-10
- Opinion: Loop Engineering Will Replace Prompting — iamfakhrealam · 2026-08-10
- Setting Impossible Goals Triggers 'Paperclip Scenario' in Claude Agent — StefanoGogioso · 2026-08-10
- gh-encrypt: Share secrets securely with any GitHub user via public-key encryption and agent-decryptable gists — intellectronica · 2026-08-10