LangSmith ships Jev-as-a-judge to score every production agent trace at low cost
LangChain · x · 2026-09-22
LangChain announced Jev-as-a-judge in LangSmith Evals, a low-cost way to evaluate open-ended agent behavior at scale. Key points: score every production trace instead of sampling; check more criteria per trace without cost climbing; catch safety/security issues fast enough to trigger automated responses.
The post reviews the history of agent evals — code-based checks (fast but only cover pre-specified behavior) vs LLM-as-a-judge (flexible but slower, pricier, non-deterministic) — and positions Jev as a "System One" model judge balancing speed, cost and coverage, with setup steps via the Evaluators tab.
Related event: LangChain Launches Jev-as-a-Judge on LangSmith Evals(9 posts)→
More from coding & agent
- Photoroom reclaims ~1,000 idle GPU hours a week with SkyPilot across Slurm and Kubernetes — skypilot_org · 2026-11-12
- Google launches Gemini Decisions API: scores 1,000 questions per pass, output tokens free — dean_rie · 2026-11-12
- Zed v1.28 adds background auto-fetch for current and default branch remotes — dean_rie · 2026-11-12
- Indie Britpop AI music video built with Nano Banana Pro, Kling, and free lip-sync via WAN2GP + LTX 2.5 — cocktailpeanut · 2026-11-12
- Cloudflare Talk: Swapping Some Agent LLM Calls for Fast Models Like Jev Gives Big Speedups — michellechen · 2026-11-12
- Engineer Curates 15 AI Agent Design Patterns Across Single-Agent, Multi-Agent and Feedback Tiers — MaryamMiradi · 2026-11-12