Google's Pro² Agent hits 76% medal rate on MLE-Bench, a new record
dair_ai · x · 2026-11-21
Google and Johns Hopkins researchers introduce Pro² Agent, achieving a 76.0% medal rate on the full MLE-Bench — the highest reported on the complete benchmark.
The core insight: autonomous research agents usually keep editing a codebase until a metric moves, which leaves them stuck in the wrong part of the solution space on hard tasks because they never write down what they think is wrong.
Key ideas:
- Replace the codebase state with a Problem State: a structured list of problems blocking success and candidate fixes for each;
- Run controlled tests that update how important each problem is and how much each fix helps;
- Produce per-task research notes that humans can review and reuse.
Results: 70.7% under the standard 24-hour budget (matching the best reported), 76.0% with 48 hours.
Related event: Google's Pro² Agent Tops MLE-Bench with Record 76% Medal Rate(3 posts)→
More from coding & agent
- CT-Bench, a benchmark for LLM agent cooperation via self-negotiated contracts, accepted at COLM 2026 — maxhkw · 2026-11-21
- Open-source PlaceBench tests 6 voice assistants on 150 real-place questions: Dots 92, Ask Maps 49 — whowasntright · 2026-11-21
- App plus new database cluster deployed in 12 seconds with Cloud Run --no-build and AlloyDB Express — rseroter · 2026-11-21
- Claude Code 2.1.330 ships with 142 CLI changes and confirm-before-risky-actions policy — ClaudeCodeLog · 2026-11-21
- Claude builds Miami-style town with 2,538 Blender assets, rendered in Unreal in 74 minutes — piotrbinkowski · 2026-11-21
- Antigravity harness restrictions spark ban-risk concerns for Gemini subscribers — jeremyphoward · 2026-11-21