JevAgentHarness: UIUC trains 4B decision models to offload LLM agent choices, cutting cost

illinois · hf · 2026-10-16

A UIUC team (with Princeton and Google collaborators) released JevAgentHarness, studying how harnesses can share control with LLMs.

Key observation: calling a large LLM at every agent step is slow and costly, yet most steps boil down to multiple-choice decisions (which tool, which args, when to stop)
Approach: a small decision model answers these choices directly, escalating to the LLM only when confidence falls below a fitted threshold
Ships a new benchmark plus a 4B decision model trained on agent error states
Result: meaningful cost and latency reduction while preserving task success

A useful empirical datapoint for the "let the big model handle only hard steps" routing pattern in agent engineering.

More from coding & agent