EngiWorld benchmark: AI agents tested across 26 engineering tools, best scores only 44.7
青稞AI · wechat · 2026-10-13
Researchers from Tsinghua, Chongqing University, UCAS, Peking University and ZhimanAI introduce EngiWorld, an agent benchmark grounded in real engineering workflows — e.g., designing a heat sink requires AutoCAD drafting, SolidWorks modeling, and Abaqus thermal analysis with dimensions, units, and physical conditions kept consistent.
The benchmark has 1,301 tasks across 26 software applications spanning CAD, CAE, CAM, BIM, EDA, and DCC: 610 GUI tasks and 691 command-line tasks, in six categories including single-software execution, visual-guided modeling, design optimization, and cross-software collaboration. Verification reopens submitted files to check dimensions, solids, materials, loads, and connections.
Seven models were evaluated on 306 stratified tasks: Claude Opus 5 led with an EngiScore of 44.7, and all seven models scored zero on 129 tasks. Cross-software workflows are the weakest point — only 6 of 168 attempts succeeded (3.6%); a proportional-import error reappeared after software restart, dooming the heat-sink thermal analysis.
More from coding & agent
- Decisions API hits GA as agents automate cloud video editing via feedback loops — dkundel · 2026-10-23
- Swarms API changelog: 8 months of streaming, multi-agent orchestration, and dynamic model pricing — KyeGomezB · 2026-10-23
- A curated library of 50+ free AI agent resources: videos, repos, one Google Doc — ifioknkem · 2026-10-23
- Vercel adds no-account deploys: drag and drop a file to get a live URL — steren · 2026-10-23
- Dev runs 5 Claude Code sessions that coordinate via MCP Agent Mail to root his network cameras — doodlestein · 2026-10-23
- Vercel ships 'vercel deploy --temporary' for ephemeral agent-built deployments — cramforce · 2026-10-23