JevAgentHarness: a 4B decision model lets cheap agents offload tool choices from the LLM
youjiaxuan · x · 2026-10-16
LLM agents call a big model at every step, yet most steps are just multiple-choice decisions like "which tool next?" The JevAgentHarness paper (arXiv 2610.21412) benchmarks and trains decision models that share control with the LLM:
- Findings: the most accurate decision model still only solved 69.5% of agent tasks; naive delegation without a fitted confidence threshold cut success by 19 points.
- Method: JevAgentHarness fits a confidence threshold per choice on the agent's own trajectories — the small decision model decides when confident, otherwise the LLM takes over.
- Training: they release AgentDecision-4B, an open 4B model trained on states agents reach after their own mistakes, solving 91.5% of out-of-distribution tasks.
Comes with AgentDecisionBench and the DecisionGym environment.
Related event: JevAgentHarness: Small Models Take Over LLM Agent Decisions(2 posts)→
More from coding & agent
- Products now sell twice: first to agents, then to humans who trust them — FinanceYF5 · 2026-10-16
- Sentry CEO shares screenshot of AI agent talking back to a human in code review — zeeg · 2026-10-16
- SWE-Duel: self-scaling adversarial arena has coding agents hide and catch bugs — Consistent_Fox_2990 · 2026-10-16
- Turn your Claude Code and Codex agent traces into training-ready SFT datasets for small open models — Rasmic · 2026-10-16
- LangChain dev: auto-generating evals from prod traces only gets you regression tests — BraceSproul · 2026-10-16
- Rebasely launches in beta: Git hosting built for humans and coding agents — handotdev · 2026-10-16