Snorkel's $30M benchmark grant spurs first real talks among eval experts
typewriters · x · 2026-10-28
The author co-hosted a small SF dinner with SnorkelAI bringing together benchmark/eval practitioners from the private sector and nonprofits. They highlight a paradox: everyone claims AI standards will drive a race to the top, yet the people actually building and maintaining benchmarks don't collaborate — without shared measurement frameworks we can't verify whether AI systems meet standards. The dinner followed SnorkelAI's $30M open benchmarking grant program and preceded the first ARC Prize Research Summit in Boston.
Related event: AI Eval Experts Gather as Third-Party Audit Ecosystem Remains Fragmented(3 posts)→
More from Companies & People
- Ex-Big Tech Dev on Why He Renamed Himself 'Applied Generalist' — and Backs the MTS Movement — signulll · 2026-10-28
- Figma Signs On as Platinum Sponsor of Open-Source Media Library Mediabunny — Vjeux · 2026-10-28
- Owning AI infrastructure lets Meta ship model variants faster, argues Ben Bajarin — BenBajarin · 2026-10-28
- Modal hires Frosina Makraduli as Europe DevRel lead, offers free credits to projects — charles_irl · 2026-10-28
- Cohere Labs hosts Oct 28 research session with Lila Sciences and Physical Intelligence scientists — tw_killian · 2026-10-28
- Developers report Apple is automating App Review, with trusted accounts approved almost instantly — rudrank · 2026-10-28