Open 4B decision model cuts agent costs 81-89% and beats the closed model it learned from

alex_verem · x · 2026-10-19

A companion post sharing the same JevAgentHarness paper: agents delegate their multiple-choice decisions (tool choice, stopping) to an open 4B model, calling the big LLM only when unsure.

Highlights: 81-89% cheaper and up to 2.5x faster on similar tasks with success within 3 points; 31ms median responses vs 809ms; the top standard-benchmark model (92.2%) solved only 69.5% inside a real agent, while the 4B model hit 91.5% — beating its teacher Jev (82.0%) — because it trained on post-mistake agent states. Limits: coding agents saved only 7-10.5%, and rule changes crashed success to 3%.

Original post →

Open 4B decision model cuts agent costs 81-89% and beats the closed model it learned from

More from coding & agent