sPTC speculatively launches tool calls during code generation, speeding agents 1-1.2x
dair_ai · x · 2026-08-25
Recommended by dair.ai, Speculative Programmatic Tool Calling (sPTC) comes from the same author as RLM, targeting the agent harness layer.
The problem: harnesses make agents wait — the model streams a code block, and tool calls inside it only run once generation finishes, so tool latency stacks on top of token generation time.
The approach: sPTC speculates on upcoming tool calls mid-generation and launches the safe ones early against a copy of the environment, overlapping tool latency with token generation; bad guesses are simply thrown away.
Results: a 1–1.2x speedup so far, described as very promising, with many more harness-layer efficiency gains possible.
More from coding & agent
- Engineer: my job went from silently thinking at a computer to constantly talking to it — KevinNaughtonJr · 2026-10-16
- Vercel Goes Full Backend: Python GA, Containers, and Daemons — cramforce · 2026-10-16
- WorldMemArena Accepted at NeurIPS: Memory Storage ≠ Memory Use — xwang_lk · 2026-10-16
- Vercel Ship Demos Container Deploys for Go, Ruby, Java and More — edwin · 2026-10-16
- TRL adds day-0 support for fine-tuning Muse Spark Open's 816B MoE with LoRA on 64 H100s — QGallouedec · 2026-10-16
- Databricks engineers land impactful PRs on day 2 with AI dev tools, vs week 3 before — Yuchenj_UW · 2026-10-16