IBM's JevAct adds runtime decision gates to agents, lifting τ²-Telecom pass rate from 39.5% to 80.7%
omarsar0 · x · 2026-10-31
IBM's JevAct: a runtime gate between agents and their tools
IBM researchers published Think Before You Act: Decision Gate for LLM Agents, keeping models and tools unchanged while adding a verification layer:
- How it works: an LLM compiles domain policies into code checks and yes/no questions. Before each tool call runs, code checks facts in tool records while a Jev-like prefill-only decision model judges the conversation. Failed calls bounce back to the agent with feedback.
- Results: on τ²-bench Telecom, Opus 4.6's three-run pass rate jumped from 39.5% to 80.7%. All eight agents improved by 16.9 points on average on Telecom; all nine improved on BFCL. Airline gains were smaller and mixed.
- Takeaway: the gains come from the harness, not the model — the framework works independently of the underlying agent model.
More from coding & agent
- 174-line mental schema + Claude Code: an agent workflow that builds entire slide decks — tianshi_li · 2026-11-01
- Stripe Link goes live in Agent Picnic to manage AI agent connections and permissions — jeff_weinstein · 2026-11-01
- BossBot books pickleball courts at midnight daily, beating the reservation rush — chrisfleck · 2026-11-01
- OpenAI's Astra 5.5 runs massive code patches just to list Devcon side events — devanshmehta · 2026-11-01
- Curated skills collection lets you run agents on your own harness and subscriptions — jasonkneen · 2026-11-01
- Workshop: Build a media agent on CPU, call image and TTS models as MCP tools — Al_Grigor · 2026-11-01