Microsoft Paper: Agents Succeed Often but Rarely Reliably

A Microsoft paper finds top agents solve 91% of tasks at least once but succeed consistently only 25% of the time. The company released ThinkingBox, a sandbox benchmark of 507 validated tasks, to measure agent reliability in real business workflows.

2026-08-23 ~ 2026-08-23 · 2 related posts

• Microsoft's Thinkingbox: best model hits only 65% pass@1 on real business workflows — dair_ai · 2026-08-23
• Microsoft Paper: Agents Succeed 91% Once but Are Reliable Only 25% of the Time — rohanpaul_ai · 2026-08-23