METR open-sources its AI agent evaluation platform for safely testing frontier models
BethMayBarnes · x · 2026-11-15
A METR researcher announced the open-source release of the org's evaluation platform, built around thorny technical challenges in safely testing highly capable agents:
- Automatically testing sandbox containment without causing security incidents
- Letting researchers securely run arbitrary agents executing arbitrary code on a cluster
- Safely managing penetration tests conducted by red-team frontier models
- Economically and accurately monitoring all inference for "Bad Stuff" — and defining what that even means
The team plans to publish more results from experiments on securing AI agents.
More from coding & agent
- Solo lawyer runs an AI-native law firm on a Claude agent team organized like company departments — prasanna_says · 2026-11-15
- Claude builds a colorful 3D city in three.js with zero assets, all code — Eric520CC · 2026-11-15
- fate 2.2.1 ships massive perf gains: up to 343x faster record deletion — cnakazawa · 2026-11-15
- schu-video-generator: Claude Code renders full music videos with no video model — CoolTopicsNow · 2026-11-15
- Ditch the agent factory: making the pull request the unit of AI coding work — railsonamaui · 2026-11-15
- Anthropic study: one agent per person performs worse than a single shared agent — vista8 · 2026-11-15