Speculative Programmatic Tool Calling Speeds Up Code Agents by 1.2x
a1zhang · x · 2026-08-26
Alex Zhang introduces "Speculative Programmatic Tool Calling" (sPTC), a harness-level trick to reduce latency in code agents. By speculatively executing tool calls parsed from partial code generations while the model is still streaming, sPTC overlaps high-latency sub-LLM/sub-agent calls with token generation. A "shadow REPL" tracks dependencies safely. Experiments show this achieves 1.2x wall-clock time speed-ups without modifying the model or agent logic.
More from coding & agent
- AI software factories hinge on the Factory Loop, not faster code generation — Pavan_Belagatti · 2026-10-16
- 'Dropping a 3,000-line PR twenty minutes after ticket assignment' — AI coding meme goes viral — facontidavide · 2026-10-16
- Self Labs launches Agent Vault to let AI agents log in and pay with credit cards — EXM7777 · 2026-10-16
- Plane Agents: triggers, not prompts, are what turn chatbots into workflow agents — JosephJacks_ · 2026-10-16
- Claude eng lead Felix Rieseberg publishes tutorial on being more ambitious with AI — danshipper · 2026-10-16
- Qwen Code Desktop v0.25.2 ships managed-agent fixes and MCP config repair — github-actions[bot] · 2026-10-16