Anthropic spent $3M on 30 months of agent evals across real company tasks
2C_ornot2C · x · 2026-07-25
- Anthropic reportedly spent $3M and 30 months studying AI agents on real tasks inside real companies, covering 1,287 tasks and 2.4M lines of code.
- The cited article argues that the results show why system design, memory, and iteration loops matter more than prompt-only workflows.
- The screenshot highlights Loop Engineering: clustering traces, building an annotation app, and using AI to monitor annotations in real time to improve sampling and acceptance/rejection decisions.
- The attached paper-style image reports a study of graph engineering for agent workflows, including substantial productivity and quality gains.
More from coding & agent
- ThoughtSpot launches SpotterCode, a hosted MCP server letting Claude Code query live instances — thoughtspotdev · 2026-10-26
- Non-developer stuck at 65% with Claude Code: rules files and CLAUDE.md weren't enough — jinguh · 2026-10-26
- HermesOBD: open-source macOS app pipes live car OBD-II data to a local Hermes agent — Teknium · 2026-10-26
- User Claims Codex Autonomously Reverse-Engineered Serum2 and Stripped Its License — Kyrannio · 2026-10-26
- colab-mcp lets AI agents control Google Colab notebooks directly — helloiamleonie · 2026-10-26
- Postiz, an open-source AI-agent social media manager, hits 37.9k GitHub stars — abhishek__AI · 2026-10-26