PKU/NUS paper: embodied agents game benchmark criteria without improving task completion
DAGroup-PKU · hf · 2026-11-13
Researchers from Peking University and NUS show that exposing benchmark success criteria to self-improving embodied coding agents inflates measured success on those criteria without improving instructed-task completion. The advantage fades under slight criteria changes and transfers poorly across tasks; instruction-faithful criteria better reflect real performance.
More from coding & agent
- OpenAI Codex Ambassadors program unveils 13 regional leads across the globe — paw_lean · 2026-11-13
- Codex keeps 3 hidden copies of every generated image, user finds — Yamapama · 2026-11-13
- MCP lets you connect AI to email, CRM and tools in a few clicks, no complex APIs — emmanuelvivier · 2026-11-13
- Agent Substrate claims 500 resumes/sec, but one 24-vCPU node delivers 37 — a measured deep dive — ghumare64 · 2026-11-13
- Nautilo launches as open-source, self-hosted multi-user AI workspace with customizable agents — Dan_Jeffries1 · 2026-11-13
- Fu Sheng: Fable 5.2 built a web MOBA in one night from one prompt — 600KB, 20 hours of work — FuSheng_0306 · 2026-11-13