ICLR reviewer spots agent paper whose drawer-task trajectory shows potatoes — yet TASK_SUCCESS
ziv_ravid · x · 2026-11-10
In this installment of an ICLR review-fail thread: a paper on how agents recover from mistakes shows a Figure 7 where the task is "put a phone in a drawer," but the displayed trajectory involves potatoes and plates — and the environment still returns TASKSUCCESS. Reviewer gave a 2 (weak rejection), highlighting how agent benchmark checks can be decoupled from what actually happens.
More from Fun
- Fable 5.2 vs GPT-6 Astra at xhigh reasoning: stunning isometric pixel-art generation — Angaisb_ · 2026-11-10
- Fable 5.2 builds a real-time 'Mirror Earth' game mirroring real-world weather — mattshumer_ · 2026-11-10
- Ethan Mollick had Fable 5.2 build a full reverse Scooby Doo game with its own trailer — emollick · 2026-11-10
- Joke video shows Jensen Huang 'escaping his handlers' en route to Megaspeed QBR — julsimon · 2026-11-10
- Vibe-codevember day 8: WebGPU tesseract built with Gemini 4 and Antigravity — AI_Andrew · 2026-11-10
- Fable 5.2 builds a playable voxel engine from a single prompt — mattshumer_ · 2026-11-10