Economist: since GPT-5, labeling disagreements are almost always the LLM being right
soumitrashukla9 · x · 2026-10-06
- Economist John Horton's hot take: from about GPT-5 onward, when humans and LLMs disagree on a data-labeling task, checking the actual disagreements shows it's almost always the LLM being right and the humans wrong — or the humans substantively disagreeing with each other.
- Implication: human labels are losing status as ground truth; model–human disagreements may be a stronger signal for dataset and benchmark construction.
More from AGI Musings
- OpenAI trains agents to swarm: multi-agent coordination could be the next scaling law — binarybits · 2026-10-06
- 90+ Claude Opus 5.5 agents find two room-temperature magnetic semiconductor candidates in 3 days — Dr_Singularity · 2026-10-06
- Paul Graham: Even the Smartest People Must Learn Taste in Problems From Experience — ycombinator · 2026-10-06
- iamtrask: AI consciousness and AI copyright are the same debate — iamtrask · 2026-10-06
- Columnist: "AI Alignment" may go down as one of humanity's most misguided ideas — examachine · 2026-10-06
- OpenAI set to publish ~400 AI-generated math proofs, and mathematicians can barely digest one — RexDouglass · 2026-10-06