Speculative Programmatic Tool Calling Speeds Up Code Agents by 1.2x
a1zhang · x · 2026-08-26
Alex Zhang introduces "Speculative Programmatic Tool Calling" (sPTC), a harness-level trick to reduce latency in code agents. By speculatively executing tool calls parsed from partial code generations while the model is still streaming, sPTC overlaps high-latency sub-LLM/sub-agent calls with token generation. A "shadow REPL" tracks dependencies safely. Experiments show this achieves 1.2x wall-clock time speed-ups without modifying the model or agent logic.
More from coding & agent
- Google Scientist on Long-Horizon Agents: Balancing Context vs. Scaffolding — AnneliesGamble · 2026-08-26
- OpenHuman, an open-source local-first AI brain with persistent memory, hits 37.9k GitHub stars — abhishek__AI · 2026-08-26
- Scale AI Employee on Data Quality: No Shortcuts to Intuition — JohnnyNi13 · 2026-08-26
- Code Diff Tool Codiff Released Version 1.11 — cnakazawa · 2026-08-26
- Hugging Face Glossary Clarifies Confusing AI Agent Terms — SergioPaniego · 2026-08-26
- AI agent suggestion bricks Surface during Linux install — mohamedmansour · 2026-08-26