Study Finds Jev Is a Competitive Multilingual LLM Judge at 1/31 to 1/61 of GPT-6.1's Cost
kalyan_kpl · x · 2026-11-14
A new paper benchmarks Jev against generative LLM judges on four multilingual benchmarks covering up to 47 language varieties and three judging formats: binary decisions, pairwise comparisons, and graded ratings.
- Jev is competitive on most multilingual judging tasks at roughly 1/31 to 1/61 of GPT-6.1's evaluation cost, but lags on knowledge-intensive answer verification.
- It accepts incorrect answers more often in low-resource languages than in English, and its probability outputs help flag errors across tasks.
More from Research
- Harvard-MIT UGG model unifies video generation, depth and camera pose in one diffusion model — burny_tech · 2026-11-14
- Hugging Face Introduces Decision Models That Output Probability Distributions Without Generating Text — ariG23498 · 2026-11-14
- New results in computable topology reduce computable structures to two theorems — jessi_cata · 2026-11-14
- BAAI's encoder-free VLM family EVE: NeurIPS spotlight, ICCV highlight, open-sourced — liuziwei7 · 2026-11-14
- Recursive Model Collapse: How It Learned to Stop Worrying and Become the Assistant — sebkrier · 2026-11-14
- AI helps unlock hidden discoveries in math genius Paul Cohen's notebooks — irrelevantusername24 · 2026-11-14