Microsoft paper: agents near-perfect at 2 steps collapse to 0-33% by step 16
rohanpaul_ai · x · 2026-09-08
A new Microsoft paper shows LLM agent reliability degrades sharply with task horizon.
- Each step carries an error probability that compounds; across 9 models, success drops as dependent steps increase.
- On ToolQA, models near-perfect on short runs fell to just 0-33% success at 16 steps.
- Step count, not context length, drives decay: shortening context steepened the decline, so trimming history isn't a fix.
- Advice: test agents at real workflow lengths, measure per-step reliability, and add checkpoints before a bad step poisons downstream results.
More from coding & agent
- OpenAI opens up agent sandboxes: BYO or pick from Cloudflare, E2B, Modal, Vercel and more — threepointone · 2026-09-11
- SocialCrawl MCP lets agents search Reddit, YouTube, TikTok, X with one API key — dooddyman · 2026-09-11
- Astra builds a surprisingly polished Catan game in three.js, reusing past UI and 3D assets — FinanceYF5 · 2026-09-11
- Open-Source Tool Highlights the Exact PDF Paragraphs Behind AI Answers — Flat-Phone-1596 · 2026-09-11
- OpenAI Codex may issue another usage reset this weekend, says Codex lead resets happen — umesh_ai · 2026-09-11
- Dev swaps gemini-3.8 for gemini-3.5-flash-lite in his MCP harness at a fraction of cost — julianharris · 2026-09-11