1.26M Blind Votes Show Humans Rate the Same Frozen Chatbots More Harshly Every Month
rohanpaul_ai · x · 2026-11-14
A new study of 1,260,967 blind votes on Chatbot Arena finds that "both are bad" votes rise 1.71% per month even on pairs of frozen models (dated snapshots, fixed open weights), meaning the models never changed — human evaluation standards for LLMs are steadily rising. An unchanging chatbot therefore appears to get worse month over month, not because it's getting dumber, but because we're getting pickier.
Related event: 1.26M Blind Votes Show Humans Are Getting Tougher on LLMs(2 posts)→
More from Research
- Open-source live-cell hardware startup Aeon applies causal modeling to cell perturbation data — MaxUnfried · 2026-11-14
- LMSYS and NVIDIA: Vera CPUs cut agentic RL episode time by 11% in Miles tests — ying11231 · 2026-11-14
- Judea Pearl: 'truncation by death' solutions follow from universal calculus, not tricks — yudapearl · 2026-11-14
- Rare 1956 photo of C.R. Rao with R.A. Fisher, plus a Fisher-Rao geometry primer — FrnkNlsn · 2026-11-14
- Researcher pushes back on Goodfire: 2.9% loss hike from ablation ≈ losing 50-90% of compute — RylanSchaeffer · 2026-11-14
- Latent feedback cuts pretraining compute by 46% in 4 lines of code — _arohan_ · 2026-11-14