Logan Kilpatrick questions AI leaderboard: why is only our model not tested in its native harness?
zacharynado · x · 2026-11-11
Logan Kilpatrick, who leads Gemini at Google, publicly questioned an evaluation setup, asking why his model is the only one not being tested in its native harness — hinting that leaderboard conditions may not be consistent across vendors.
More from Models
- Alleged Gemini 4 Carbon builds 92-part 3D Paris physics lab from one image (unverified) — theREALmarvin · 2026-11-11
- GBENCH: OpenAI's GPT-6 Astra tops unsaturated coding benchmark with 83.3, sweeping top 3 — teortaxesTex · 2026-11-11
- Artificial Analysis launches Intelligence Index v5 at San Francisco event — ArtificialAnlys · 2026-11-11
- Fable 5.2 video generation via code appears to lack nudity filters in JS — mickeyxfriedman · 2026-11-11
- Lenny: Designers name Claude most often, Anthropic winning hearts and minds — lennysan · 2026-11-11
- LLMs Aren't Fundamentally Discrete: Tokens Are Just Readouts of a Continuous Process — Sauers_ · 2026-11-11