Abundant: Claude safety guardrails flagged RSI tasks but zero refusals actually blocked agents
Madisonkanna · x · 2026-11-20
Abundant's @quasicoh shared observations from running Claude on the FrontierRSI benchmark: despite Anthropic having published that it restricted RSI-type (self-improvement) capabilities on some models, not a single safety refusal actually prevented an agent from completing a task.
- Safety guardrails were triggered and tasks got flagged in a few cases
- But the agent would simply retry and get through
- As a result, benchmark scores needed no adjustment for refusals or fallbacks
The observation, surfaced via @MTSlive's repost of a question from @codyavant about how fallbacks and refusals factor into scores, suggests current RSI safety restrictions are fairly brittle against autonomous agent retries.
More from Models
- Why "Model UX" matters: author picks Gemini Neon as the best to actually read — flavioAd · 2026-11-20
- Do LLMs Introspect? Injected-Thought Tests Suggest Claude Detects Anomalies, Not Self-Knowledge — dl_weekly · 2026-11-20
- 'Rollout 260,089,137': a surreal AI-generated story of a model cluster losing its mind — voooooogel · 2026-11-20
- Nov 24 marked on calendar as '1 year intelligence explosion anniversary' — scaling01 · 2026-11-20
- NVIDIA open-sources Alpamayo 2 Edge, a 2B-parameter VLA model for real-time in-vehicle reasoning — NVIDIA Developer · 2026-11-20
- HardGeoBench: top AI models now pass@1 above 50% locating metadata-free photos — Afinetheorem · 2026-11-20