Jev-as-a-Judge cuts evaluation cost 43% with 1% accuracy loss

A CMU paper proposes Jev-as-a-Judge, cascading cheap decision-only judge models with frontier models only for uncertain cases, saving 43% cost at 1% accuracy loss.

2026-09-23 ~ 2026-09-24 · 2 related posts

• CMU Paper: Decision-Only LLM Judge Matches SOTA at 0.36% of the Cost via Cascade — CarnegieMellonU · 2026-09-23
• Jev-as-a-Judge: cheap judge cascade keeps 99% of GPT-6 accuracy at 57% of the cost — dair_ai · 2026-09-24