Amazon Research: Improving LLM Judge Accuracy with Ising Models
DocXavi · x · 2026-08-29
Amazon Science published research on whether to trust LLM judges when they agree.
- Context: Shared prompts, model families, or training lineage can make a majority opinion appear stronger than it is.
- Method: Introduces a dependence-aware aggregation approach using Ising models to account for correlations between judges.
- Results: The method improves accuracy by 9–14% over weighted majority voting.
- Insight: Blind trust in consensus can be misleading; sophisticated aggregation strategies are needed to correct biases.
More from Research
- Cohere Labs world model course session 4: DreamerV1-V4 architectures in one go — Cohere_Labs · 2026-10-16
- GPT-6.1 Sol audit finds 6% of arXiv papers had false main theorems, down to 1.5% recently — nitarshan · 2026-10-16
- One tiny word changes the conversation: how LLMs pick up on 'just' — StellaLisy · 2026-10-16
- SEAR: first large-scale controlled study of self-evolving robot agents, 7 LLMs, 330 tasks — xiye_nlp · 2026-10-16
- After Claude-assisted elliptic curve records fell, EC Rank challenge opens 156 record slots — Jsevillamol · 2026-10-16
- Oxford neuroscientist swaps fetal MRI for ultrasound + ML models — AnnaCiaunica · 2026-10-16