A 4B model trained on one GPU in 2 hours beats Jev 1.13 at agent decisions, 86.5% vs 80.6%
ivan_bezdomny · x · 2026-11-07
Developer MaziyarPanahi trained a 4B "tiny decider" on a single GPU in 2 hours using Jev's newly released 720K agent-decision rows (5 datasets, 1.8B tokens of tool calls, agent traces, support, routing, policy checks), with no Jev answers in training.
- Overall: his 4B scores 86.5% vs Jev 1.13's 80.6% on 480 unseen test rows
- Per-task: agent tool choice 90.8% vs 82.5%; support tool choice 86.7% vs 90.0% (Jev wins); agent next step 85.0% vs 74.2%; support next step 83.3% vs 75.8%
Takeaway: with public labeled decision data, a small open-weights model can rival larger models at agent routing decisions — cheaply, and kept in-house (e.g. hospital settings).
More from coding & agent
- Indie dev builds Calorie Cub, an agent-first calorie logger, after MyFitnessPal's MCP broke — Odd_Apricot_3402 · 2026-11-07
- Agent security swarms: shared-wallet bug bounty hunting with zk proofs-of-exploit — banteg · 2026-11-07
- The Agentic Web: a personal AI assistant that operates every website you visit — JnBrymn · 2026-11-07
- Agents have wallets: BoundlessHQ adds pay-per-token stablecoin streaming payments — banteg · 2026-11-07
- Testing shows coding harnesses can be harmful or hugely helpful depending on the model — JoshPurtell · 2026-11-07
- Tip: Run /prototype on Small UI Changes to Get Several Options to Pick From — mattpocockuk · 2026-11-07