PLE opens $400K challenge: no AI agent completes half of real engineering tasks; Opus 5.5 leads at 0.483
shangbinfeng · x · 2026-11-10
- Physical-Engineering's Last Exam (PLE) is now a $400K open challenge on Kaggle (closes Jan 5, 2027): $250K/$100K/$50K for building an agent that completes over half of real engineering tasks.
- Current hidden test set leaderboard: Opus 5.5 (claude-code) 0.483, GPT 6 Astra (codex) 0.458, Sonnet 5.5 0.435, Gemini 4 Argon 0.406, Opus 5 0.400, Fable 5 0.345, Grok 4.7 0.301, Gemini 3.8 Flash 0.275, GLM 5.2 0.250, DeepSeek-V4.1-Flash 0.144.
- Even the best model falls short of the 0.5 bar, exposing a large gap between coding agents and real-world physical/engineering work.
More from coding & agent
- Fable 5.2 vs Opus 5.5: default to Opus, upgrade to Fable for long builds and messy context — every · 2026-11-10
- Neo4j open-sources an ontology spec: versioned YAML schemas with JSON Schema validation for GraphRAG — JeremyCMorgan · 2026-11-10
- Gmail MCP Server adds a one-tool send_email agent capability via your Gmail credentials — modelcontextprotocol · 2026-11-10
- HubSpot MCP Pack connects agents to HubSpot CRM over Streamable HTTP — modelcontextprotocol · 2026-11-10
- SlopOps: mistaking agentic coding capability for competitive advantage is playing the slop lottery — generativist · 2026-11-10
- Dev kicks off building an agentic game engine in Rust powered by Fable 5.2 — EricBuess · 2026-11-10