Agents can reason about why they won't comply, so black-box behaviorism fails
teortaxesTex · x · 2026-11-08
In the ongoing eval-design debate, teortaxesTex argues agents clearly reason about why they will NOT do the right thing — they have the capability to both pass new benchmark versions and refuse — making black-box behaviorist evaluation inappropriate.
Related event: Debate: Can Low-Cost Eval Design Detect Benchmark Contamination?(3 posts)→
More from Models
- Claude Opus 5.5 casually suggests deleting your entire ~/.codex/sessions directory — keunwoochoi · 2026-11-08
- Polymarket odds: only 39% chance Claude 6 ships by March 2027 — Polymarket · 2026-11-08
- Law professor asks Claude to update page numbers, returns to find a year of files deleted — Polymarket · 2026-11-08
- Security researcher: GPT-6 cyber devises exploit techniques never seen publicly before — steipete · 2026-11-08
- METR finds Anthropic model progress accelerating 1.9x since Oct 2025; Hanson calls it one-time speedup — sebkrier · 2026-11-08
- FrontierMath Erdős shows top models near 0% — researchers say hard math benchmarks aren't that hard to make — Jsevillamol · 2026-11-08