Cat Wu, ex-Claude Code product lead now at OpenAI: 10 great evals beat hundreds
aakashgupta · x · 2026-11-13
Cat Wu, who ran product for Claude Code and now leads core product at OpenAI, says most people over-obsess about eval theory — axial coding, open coding, generating 100 test cases.
Her core claim: you don't need hundreds of evals; building 10 great ones matters more. Evals help teams quantify the goal and see what's missing, and the real value is how you team up around them. The post also links a learning roadmap covering theory, first principles, practical examples, Braintrust and Arize.
More from coding & agent
- AmphoraDB launches serverless databases for agent swarms, paid per statement in stablecoins — banteg · 2026-11-14
- With 90% AI-generated code, Limestone gates merges on its V.U.E. three checks — alex_verem · 2026-11-14
- Ford dealership veteran pays for an AI system doing the work of 12 salespeople — aryanXmahajan · 2026-11-14
- One prompt gets AI to integrate WebMIDI so you can play your MIDI keyboard in the browser — alexanderchen · 2026-11-14
- Static model routers fail on long tasks — routing must be a dynamic handoff policy, dev argues — petrusenko_max · 2026-11-14
- MIT CSAIL Shares Programmer Wisdom: "There's Already a Library for That" — MIT_CSAIL · 2026-11-14