Just 21 annotators (6.5%) provide half the votes in Anthropic-HH-RLHF
rishabh16_ · x · 2026-09-14
Shuvom Sadhuka shared a key observation from his blog on alignment datasets: a small fraction of annotators provide most votes in common alignment datasets.
- In Anthropic-HH-RLHF, just 21 (6.5%) annotators provide 50% of the votes
- He proposes measuring the sensitivity of model rankings to dropping top voters vs. dropping an equal number of votes at random
- Comes with a thread and full blog post
Same research project as the LMArena 0.003% sensitivity finding, questioning the statistical basis of current alignment practice.
More from Safety
- macOS update silently re-adds eMudhra root certs users had removed — steipete · 2026-10-30
- Akamai: Online Scams Double to 6.4M, Overtake Phishing as Attack Windows Shrink to 22 Days — shashib · 2026-10-30
- Anthropic paper: 0.1% pretraining poison plants RCE backdoor in 4B CLI agent — omarsar0 · 2026-10-30
- Cryptographer Matthew Green slams Anthropic's 'model escape' framing of agent worm research — matthew_d_green · 2026-10-30
- Pope Leo calls to 'disarm' AI and urges UN to ban lethal autonomous weapons — The Verge AI · 2026-10-30
- 100 open-source AI security tools challenge kicks off with DirectGuard, a sub-ms prompt injection guard — Venom943 · 2026-10-30