LangChain's Jev evaluator cuts agent eval score variance by up to 913x at 1/80th the cost
LangChain · x · 2026-09-20
- LangChain introduces Jev-as-a-Judge, an agent eval method that returns typed answers instead of generating text like LLM judges
- On continuous scoring, Jev's variance was 92–913x lower than GPT-5.6 Luna, Terra, and Claude Sonnet 4.6
- It averaged 0.44s and $0.00035/call — $0.34 total vs $28.17 for Claude
- Authors caution results are early but say it points to a compelling new direction for agent evals
Related event: LangChain Launches Jev-as-a-Judge on LangSmith Evals(9 posts)→
More from coding & agent
- Cloudflare Talk: Swapping Some Agent LLM Calls for Fast Models Like Jev Gives Big Speedups — michellechen · 2026-11-12
- Engineer Curates 15 AI Agent Design Patterns Across Single-Agent, Multi-Agent and Feedback Tiers — MaryamMiradi · 2026-11-12
- Google Antigravity rolls out browser use, closing the loop from research to coding to live verification — rseroter · 2026-11-12
- Prettier 3.10 ships new Flow/JS syntax and a Zig-powered Yuku plugin — Vjeux · 2026-11-12
- YC-backed Graphide claims CI runner 10-20x faster than GitHub on 24 repos — ycombinator · 2026-11-12
- Lessons from a 1600-page transcription: general thinking resists staying organized — fkasummer · 2026-11-12