Using fast decision models for evals: 200ms judge beats slow LLM-as-judge
holdenmatt · x · 2026-09-20
The author suggests using Jev for evals: since evals are essentially decisions (binary or numeric rubrics) and run well in parallel, a fast/cheap decision model can replace slow, expensive LLM-as-judge setups. If an eval takes only 200ms, you could even evaluate live on production traffic instead of running offline.
More from coding & agent
- Swarms Cloud changelog: Auto Agent Builder, MCP, and full agent execution observability — KyeGomezB · 2026-09-20
- Using Jev as a 75ms 'subconscious' filter to cut agent context bloat for under half a cent a day — Obvious_Unicorn · 2026-09-20
- Scoble says an AI agent wrote every word of an entire book — Scobleizer · 2026-09-20
- "Look at your data": dev mocks reflex to spin up another agent — chrisalbon · 2026-09-20
- Evaluating 7 Models Across Claude Code, Codex, and Pi: Harness Choice Drives Cost, Not Success — CShorten30 · 2026-09-20
- Dev builds MCP middleware that scrubs personal data before it reaches the AI's context — Danielloesoe · 2026-09-20