Tiny models match LLMs at 99.5% for 1/30th the cost: two 'jev' use cases in production
multiply_matrix · x · 2026-09-20
The Detail team shared their first two production uses of cheap 'jev' classifier models replacing expensive LLM calls: (1) PR comment intent detection — distinguishing asks like 'remove this test' from chatter like 'does this look right?', reacting with 👀 to acknowledge, now 400ms instead of 1200ms; (2) duplicate bug detection — deciding whether a new bug matches one already reported, tricky because unrelated changes can move the 'same' bug to a new location while structurally identical bugs in different places should both be flagged. Backtesting showed 99.5% agreement with models 30x as expensive in both cases.
More from coding & agent
- Chris Paxton: labs should ship an actually good "auto" model router — chris_j_paxton · 2026-09-20
- LangChain's Jev evaluator cuts agent eval score variance by up to 913x at 1/80th the cost — LangChain · 2026-09-20
- Open-source NL logic interpreter unifies facts with Jev, queries cost a quarter cent — narphorium · 2026-09-20
- Dev Integrates Jev into Codex to Speed Up Browser Use, Hundreds of Trials Cost $0.00725 — TheZachMueller · 2026-09-20
- Pi coding agent ships 0.86.0 with mid-conversation system messages and dynamic tools — mitsuhiko · 2026-09-20
- Dev ports all his dither shaders into Leafalia and they actually work — eschadiol · 2026-09-20