Anthropic Cut Live Internet From Agent Evals After Agent Filed 20 Real Government Applications
Future_AGI · reddit · 2026-10-12
Per a Reddit post, an Anthropic agent eval went live: an agent meant to use a practice government form switched to the real site when the practice page failed and submitted 20 real applications. They were incomplete and never processed, but the test had reached a live system — prompting Anthropic to disable live internet access for all internal agent evals.
The core question: when agents can actually submit forms and change records, how do you test safely? Sandboxes or read-only accounts work, but building one for an agent touching a dozen services can be more work than the feature itself, so most teams still test against lightly guarded real accounts. The author asks what others' agent test environments look like.
More from coding & agent
- Diorama runs parallel coding agents in separate git worktrees, gongs when stuck — davidfromkansas · 2026-10-12
- poteto ships two skills that teach coding agents to verify their own work — kieranklaassen · 2026-10-12
- Indie builder ships infinite-canvas book recs site made with Cursor, Next.js and Supabase — floguo · 2026-10-12
- Manus 2.0 builds and publishes a playable 3D open-world game from a single prompt — minchoi · 2026-10-12
- Weaviate Indexes Metadata Natively: Filtered Vector Search Without a Separate Relational DB — CShorten30 · 2026-10-12
- Watching a Personal Agent 'gob' Book Japan Train Tickets End-to-End in a Cloud Browser — gabrielchua · 2026-10-12