New RWE-bench: Best AI passes just 37% of real engineering tasks
ZeroCool86 · reddit · 2026-10-28
Xplore released RWE-bench, 20 mechanical, electrical and aerospace tasks written by practicing engineers — including a diesel crankshaft that must survive 2,400 rpm, DDR3 memory bus routing on a real FPGA board, and CFD on a turbine vane. Instead of model-based grading, a program checks every design against every requirement and runs physics solvers where needed.
- Scores: GPT-6 Astra leads at 37%, Opus 5.5 29%, GPT-6.1 Sol 25%, at $10 per run; Astra and Opus cost about 6x more per run for 12 and 4 extra points.
- Human comparison (author's rough estimate): an average person passes none; an engineering graduate might pass 1-2 tasks in their own field after a month; a specialist passes about a third of tasks in their own field at roughly a week each. The model covers all three fields in hours per run.
- Argument: code improved fast because answers are cheap to verify; physical engineering has a much longer feedback loop. A scoreboard exists now, and programmatic grading matters more than who's on top.
Related event: RWE-bench: best AI models pass only 37% of real engineering tasks(2 posts)→
More from Models
- Cursor user complains it silently switches to Grok, which fails basic tasks — kylegawley · 2026-10-28
- Liquid AI releases LFM3: 2B open-weight multimodal family with 256K context — omarsar0 · 2026-10-28
- Musk says Grok Bot keeps improving, calls Vault feature a game changer — beffjezos · 2026-10-28
- Claude refuses to gather legitimately public information, sparking guardrail backlash — mkheck · 2026-10-28
- DesignArena launches Code to Video Arena: models direct 15-45s films purely in code — yshan2u · 2026-10-28
- ChatGPT's 'yes, but with an important clarification' often precedes pure nonsense — Crixusgannicus · 2026-10-28