Pass the remaining deadline budget to each agent step instead of fresh timeouts
arpit_bhayani · x · 2026-11-17
arpitbhayani shares a first-hand engineering lesson: for multi-step agents, pass the remaining deadline budget to each downstream call rather than giving every step a fresh timeout — one change that noticeably improved one of his systems.
- Core insight: timing out only stops the caller from waiting, not the work. JavaScript's Promise.race() doesn't stop the loser; Go's context is cooperative and ignored ctx.Done() keeps running; Python's asyncio tothread() isn't forcibly stopped; Rust work can't be aborted once started.
- Three separate events: the deadline passing, a cancellation request, and the work actually stopping — handle them distinctly.
- Idempotency keys: tied to the logical operation, enforced atomically server-side, bound to original request parameters.
- Capacity math: at 100 req/s with 30s work and a 1s caller timeout, stopping work at timeout holds 100 active operations; letting it run holds 3,000.
Takeaway: cancellation is resource-consumption control, not just error handling — audit your systems.
More from coding & agent
- Tiny Shenzhen: a 17M-person city rendered in your browser, open source — vincent_koc · 2026-11-17
- Cloudflare open-sources Kitesurf, a Chromium-free browser engine for AI agents — shoumikgoswami · 2026-11-17
- Claude Code demolished Microsoft's decade-long VS Code strategy in under 6 months, says Gergely Orosz — rickasaurus · 2026-11-17
- Personal agent checks email every 5 minutes on DeepSeek V4.1 Flash at near-zero cost — alexcovo_eth · 2026-11-17
- AI-generated code passes all tests but is riddled with opaque contracts, finds Stanford professor's review — RobbWiller · 2026-11-17
- Riley Brown: After 50+ DMs, Nothing Beats Claude Code Projects Right Now — angrypenguinPNG · 2026-11-17