JevAgentHarness: 4B Decision Model Cuts Agent Cost 14.2% With Zero Task Failures
dair_ai · x · 2026-10-18
A UIUC/Princeton/Google paper introduces JevAgentHarness, which delegates an LLM agent's multiple-choice decisions (tool, args, stop) to a distilled open 4B decision model trained on Jev's answers over the agent's own trajectories—including states reached after its own mistakes. With a confidence threshold fitted per-choice, tool and argument cascades cut cost 14.2% across 500 tasks without changing any outcome; a fixed 0.5 threshold lost 21.5 points. On unseen tasks the agent hits 91.5% vs 82.0% with Jev; on familiar tasks 4B models make agents 81.1–89.0% cheaper and up to 2.5x faster with no 3+ point loss. Notably, decision models score only 69.5% on isolated questions, and naive delegation costs 19 points—threshold fitting is what makes it work. Models and harness will be open-sourced.
Related event: JevAgentHarness: 4B Model Takes Over Agent Decisions, Cutting Costs 14.2%(3 posts)→
More from coding & agent
- OpenCode hacked: API keys stolen, users told to revoke; gateway egress control as defense — pjausovec · 2026-10-18
- Perplexity open-sources SPACE: a Firecracker microVM sandbox for AI agents — AravSrinivas · 2026-10-18
- Running 20 concurrent low-latency regional-language voice agents on a single L4 GPU — DistrictUnique5259 · 2026-10-18
- Claude Code mods: event hooks that preview risky shell commands before execution — EricBuess · 2026-10-18
- fchollet: agents now write ~100% of code, yet corporate bottom lines aren't budging — fchollet · 2026-10-18
- Browser MCP Defense rule blocks external Codex agent tool calls before execution — HankYeomans · 2026-10-18