Microsoft paper: agents near-perfect at 2 steps collapse to 0-33% by step 16
rohanpaul_ai · x · 2026-09-08
A new Microsoft paper shows LLM agent reliability degrades sharply with task horizon.
- Each step carries an error probability that compounds; across 9 models, success drops as dependent steps increase.
- On ToolQA, models near-perfect on short runs fell to just 0-33% success at 16 steps.
- Step count, not context length, drives decay: shortening context steepened the decline, so trimming history isn't a fix.
- Advice: test agents at real workflow lengths, measure per-step reliability, and add checkpoints before a bad step poisons downstream results.
More from coding & agent
- Spring AI 2.1.0-RC1 adds tool-call confirmation, pausing agent loops before real-data access — therealdanvega · 2026-10-26
- 250 Builders Hack With Codex at Berlin Community Hackathon Ahead of OpenAI DevDay — paw_lean · 2026-10-26
- Building AI bots on raw APIs? Devs say models are 'dumb without the harness' — madeofAjala · 2026-10-26
- HarnessMM cuts agent tokens per task by up to 88% while raising success rates — omarsar0 · 2026-10-26
- Microsoft demos intent-classifying agents built with Copilot App and Foundry, deployable across Teams and Outlook — marlene_zw · 2026-10-26
- Looking for a wearable mic with best battery life that feeds recordings to personal agents via MCP — altryne · 2026-10-26