Princeton and Google paper: 4B decision model cuts agent costs by up to 89%

rohanpaul_ai · x · 2026-10-19

A UIUC, Princeton and Google paper introduces JevAgentHarness, built on the observation that most LLM calls in an agent harness are just multiple-choice decisions like which tool to call next.

Key findings

The decision model most accurate on isolated questions solved only 69.5% of real agent tasks, and handing over every tool choice without a fitted confidence threshold lowered success by 19.0 points.
The harness lets a fast decision model answer each multiple-choice question, but only accepts answers above a confidence threshold fitted on the agent's own runs; otherwise the LLM decides.
The authors also trained AgentDecision-4B, an open 4B model on states agents actually reach (including after their own mistakes). It solved 91.5% of tasks vs 82.0% for JEV.
On training-like tasks, this made agents 81.1%–89.0% cheaper and up to 2.5x faster while losing no more than 3 points of success.

Models, data and harness will be open-sourced. Practical takeaway: to offload routine agent choices to a small model, train it and set its threshold on your own agent's trajectories.

Original post →

Princeton and Google paper: 4B decision model cuts agent costs by up to 89%

More from coding & agent