Google's Physical-Engineering's Last Exam: all 14 frontier models score below 50%
dl_weekly · x · 2026-11-12
Google AI & Infrastructure, in collaboration with Mercor, released Physical-Engineering's Last Exam (PLE), a new benchmark and open competition (leaderboard at ple-bench.com) testing frontier models on mechanical, electrical, civil and structural engineering tasks.
Why existing benchmarks fall short: most test textbook-style questions, single tool calls, or small self-contained problems — frontier models already score above 90% there. None test whether an agent can produce safe, code-compliant, cost-effective designs under a licensed engineer's review.
Results: across 85 long-horizon engineering tasks, all fourteen frontier models scored below 50%, and only 41.3% of the leader's deliverables passed every safety check. Google released 29 public tasks plus grading tools. PLE complements Humanity's Last Exam (general knowledge) and Agents' Last Exam (digital professional work), targeting work that helps build the physical world.
More from Models
- Mystery Google model 'Gemini Decision 1' spotted on model tracker — imjustnewatai · 2026-11-12
- AI in 24 hours: Gemini 4 Carbon, Claude Whiteboards, Qwen4-Max preview, Cursor multiplayer agents — minchoi · 2026-11-12
- DeepSeek V4.1 Flash runs at 20 tok/s on a single RTX 3090 with 64GB RAM — casper_hansen_ · 2026-11-12
- Googler casually mentions 'our first Gemini 4 models' — strongest public hint yet of Gemini 4 — Div_pradeep · 2026-11-12
- Robot build test: GPT-6 Astra stabilizes two-board structure, Claude Fable 5.2 falls short — qinzytech · 2026-11-12
- Zvi dissects Anthropic's latest system card, disputing the autonomy threshold call — Don't Worry About the Vase (Zvi) · 2026-11-12