Delip Rao: finetuned judge models like Jev won't deliver true judge diversity
deliprao · x · 2026-09-26
AI researcher Delip Rao called typesafeai's Jev a pivotal product and design direction, but cautioned that all Jev-like models appear to be finetunes of existing LLMs. That means the resulting "judges" make similar mistakes and reinforce the same biases in production — so switching to this version of Jev doesn't help. He argues we need to build such models from scratch to get true diversity of competent judges.
Related event: Study: Cheap LLM Judges Make Correlated Errors, Cascades Barely Help(8 posts)→
More from Models
- Micius-Flash: 27B Multimodal Model for Hardware and Semiconductor Agents — zhiman-ai · 2026-11-11
- Baidu says RSI auto-optimized ERNIE inference, 6.24x decode speedup on Kunxin P800 — xiaohu · 2026-11-11
- Baidu's ECHO: multi-token prediction reusing the last layer as a drafter — xiaohu · 2026-11-11
- ERNIE 5.5 Flash Preview: 280B-param multi-router MoE activating only 14B per token — xiaohu · 2026-11-11
- ERNIE 5.5 recreates the Pelican test: 117-second animation drawn frame-by-frame in Python — xiaohu · 2026-11-11
- 35,000+ Grok-built apps exposed patient data, plaintext passwords, and sales pipeline — The Decoder · 2026-11-11