First double-blind eval of a proprietary model using a private benchmark ships
KLdivergence · x · 2026-09-16
KLdivergence announces a project with AVERI, OpenMined, MLCommons and Singapore's AISI: the first double-blind evaluation of a proprietary model using a private benchmark — model provider can't see the test items. The work advances secure evaluation as a field, preventing benchmark overfitting. He also celebrated the birth of his son.
More from Safety
- Investigation Claims EA Donors Funded Guardian's AI Coverage: All 6 Participants Paid by Same Ecosystem — beffjezos · 2026-09-16
- AI 2027 authors pitch Plan A: delay superintelligence to 2040 with fully open AI research — Turn_Trout · 2026-09-16
- EA's media capture and doomer headlines skew public AI perception, argues Nahom Sisay — NathanpmYoung · 2026-09-16
- Scholars refuse AI lab jobs, warning independent AI eval experts are too scarce — RishiBommasani · 2026-09-16
- AI's hardest problems need democratic deliberation — and independent experts — RishiBommasani · 2026-09-16
- Paper: upsampling alignment discourse in pretraining cuts misalignment from 45% to 9% — TuhinChakr · 2026-09-16