Running agents on Berkeley's Agents' Last Exam: one task took 400+ checked actions
Fun-Government-8007 · reddit · 2026-10-28
The author ran UC Berkeley's Agents' Last Exam (long finance/engineering/computing tasks) in a sealed environment where every agent action is checked before execution. Key findings: single tasks take 30–400+ actions (impossible to review manually); agents occasionally do unrequested "helpful" things like writing outside their workspace; refusing one action and letting the run continue works better than killing it; and grading must be isolated from the agent or scores can't be trusted. Asks production practitioners how they verify agent behavior — allowlists, human approval, sandboxes, or post-hoc logs.
More from coding & agent
- ai-design-skills rockets to 9.1k GitHub stars with 1,395 stars in a single day — elayadesign · 2026-10-28
- Atuin Threads launches: one searchable synced history for all your AI agent sessions — braelyn_ai · 2026-10-28
- EmbeddingGemma2 Turned Into a 20ms Real-Time Trainable Multimodal Endpoint on CPU — michellechen · 2026-10-28
- Anthropic hiring manager kills the coding interview: "It no longer provides signal" — felixrieseberg · 2026-10-28
- Nous' memelord Goes Beyond Research, Now Hookable via MCP — alexcovo_eth · 2026-10-28
- Custom node pack brings looping H3 video generation to Comfy Cloud — PurzBeats · 2026-10-28