OpenAI's Logan Kilpatrick: AI product teams should spend >25% of time on benchmarks
OfficialLoganK · x · 2026-09-22
Logan Kilpatrick of OpenAI advises teams building AI products to spend over 25% of their time creating benchmarks and lobbying model labs to care about them. Building custom evals and getting labs to optimize for them, he argues, is the easiest way to accelerate a company's progress.
More from coding & agent
- Pigment Agents now reads and posts in Slack, inheriting user permissions — mattturck · 2026-11-12
- Map of the agent economy: 50+ startups across payments, spend control, identity, memory — venturetwins · 2026-11-12
- OpenCode launches Browse, an on-device browser agent with local logins — Saul_Loveman · 2026-11-12
- VentureTwins maps the B2A economy: dozens of agent infra and app companies in first series installment — venturetwins · 2026-11-12
- Google releases 76-page 'Agents Companion' whitepaper on agent engineering — mdancho84 · 2026-11-12
- Spinning up containers for 20-line agent tools is backwards: Safescript compiles provable, 50ms-deploy apps — uriwa · 2026-11-12