sPTC: Speculative Programmatic Tool Calling Speeds Up Agents by 1~1.2x

On 08-25, author Alex Zhang released Speculative Programmatic Tool Calling (sPTC), a harness-level optimization technique for agent frameworks that use code as their primary action space, such as RLM and CodeAct. Experiments show it can speed up agents by roughly 11.2x.

Confirmed

• Core mechanism: while the LLM streams code REPL call blocks, upcoming tool calls are speculated on and queued for execution ahead of time, overlapping tool call time with token generation and REPL execution to reduce user wait time
• The design draws inspiration from CPU speculative execution and LLM speculative decoding, making it a harness-level optimization
• Best suited for systems where code is the main action space, especially RLM-style agent frameworks

Why it matters

• In a traditional harness, agents must serially wait for model streaming output and tool execution; sPTC overlaps the two to cut end-to-end latency—a practical speedup that requires no changes to the model itself
• Accounts such as dair-ai and CShorten30 reshared and recommended it, drawing notable community attention; related discussions also mentioned speculative collaboration and other extensions

2026-08-25 ~ 2026-08-25 · 5 related posts

• [source] Speculative Programmatic Tool Calling Overlaps Code Gen and LLM Inference — a1zhang · 2026-08-25
• [source] sPTC speeds up agents via speculative tool calling — a1zhang · 2026-08-25
• sPTC: Speculative Tool Calling Overlaps Execution With Token Generation — CShorten30 · 2026-08-25
• sPTC speculatively launches tool calls during code generation, speeding agents 1-1.2x — dair_ai · 2026-08-25
• Speculative Tool Calling Overlaps Gen and Execution for Faster Agents — CShorten30 · 2026-08-25