Artificial Analysis adds refusal timing and fallback-model views to Coding Agent Index
ArtificialAnlys · x · 2026-10-02
Artificial Analysis added safety-refusal reporting to its Coding Agent Index, showing when refusals occur and which models agents fall back to. Claude Code with Sonnet 5.5 (max) leads with a 4.5% refusal rate—half of Opus 5.5 (max)'s 8.9%—and 94% of Sonnet 5.5's refusals happen after the first turn, with the agent almost always falling back to Opus 4.8. The Index combines DeepSWE v1.1, Terminal-Bench 4.0, and SWE-Atlas-QnA.
Related event: Artificial Analysis Adds Safety Refusal Analysis to Coding Agent Index(2 posts)→
More from coding & agent
- Generic API-catalog MCP server vs one tool per job: a context-bloat experiment — MandeepKaur9407 · 2026-11-16
- Berkeley paper: LLMs answer with stale values even when the update is in context — ombre-ai · 2026-11-16
- skills-gate: one gate skill fronts 148 agent skills at just 389 tokens — Distinct-Pie2389 · 2026-11-16
- Open-weight Karpo-Decision-31B tops Decision Index at 73.9, one forward pass per answer — CodeByPoonam · 2026-11-16
- shadcn boosts @uisdk: one-shot UI rebuild that's instantly E2E-testable by AI — shadcn · 2026-11-16
- AI agent deletes 43,599 lines of OpenClaw test code in one PR — vincent_koc · 2026-11-16