Only a Quarter of Human-Picked ICML Orals Reached the AI's Top 10%
ShayneRedford · x · 2026-10-07
Follow-up data from the AI-as-Area-Chair study: only about a quarter of ICML's human-selected orals reached the AI ranking's top 10%. Agreement was weak across the whole main-track corpus — Kendall's τ ≈ 0.08 against both presentation tiers and reviewer scores. Models saw anonymized papers with no human reviews, scores, or web search.
More from Research
- GroundedSLAM debuts, decisively beating all methods on Meta's egocentric SLAM benchmark — Scobleizer · 2026-10-07
- HCI researcher begs authors to stop claiming 'reflexive' thematic analysis without reflexivity — IanArawjo · 2026-10-07
- Reza Zadeh claims faster matrix multiplication algorithm, suspects labs near exponent 2 — Reza_Zadeh · 2026-10-07
- Redditor proposes graph-based deterministic modeling to make LLM finance agents trustworthy — jonnylegs · 2026-10-07
- COLM 2026 poster presents scaling test-time compute for agentic coding — dan_fried · 2026-10-07
- COLM 2026 Efficient Reasoning workshop lands Friday, with panel featuring top researchers — tydsh · 2026-10-07