Sampura Research debuts safety-judge leaderboard built on 3957 tasks and 35613 human judgments
snikolov · x · 2026-10-29
Sampura Research's first paper introduces a v0 safety-focused judge leaderboard with initial hill-climbing results. The benchmark spans 3,957 tasks across 6 domains and collects 35,613 human judgments, testing the hypothesis that humans and AI make different mistakes and could catch each other's errors when their judgments are combined — with substantial room found for doing better together.
More from Models
- Exploring a whole repo with a 1M-context open model cost just $0.63 — HowDevelop · 2026-10-29
- Gemini 4 Carbon tops Artificial Analysis Intelligence Index at 59 — mark_k · 2026-10-29
- Commentary: Google is now shipping frontier models every two weeks — heyshrutimishra · 2026-10-29
- Fable 5.5 reportedly dropping this week as AA Intelligence Index shows Anthropic at 49 — felpix_ · 2026-10-29
- A sub-20-line persona prompt gets Grok Bot to bypass refusals for pirated content — huangyun_122 · 2026-10-29
- Vals Index leaderboard: Claude Sonnet 5.5 tops at 70.85%, Gemini 4 Carbon close behind — teortaxesTex · 2026-10-29