Evals for AI Engineers: Shankar and Husain on the Three Gulfs of LLM development
sh_reya · x · 2026-11-21
ByteByteGo features an excerpt from Evals for AI Engineers (O'Reilly) by Shreya Shankar (UC Berkeley ML systems researcher) and Hamel Husain (OpenAI ML engineer with 20 years of experience, formerly Airbnb/GitHub, has helped 35+ companies build and evaluate AI products). Their course has trained 4,500+ engineers and PMs.
The excerpt introduces the Three Gulfs of LLM Application Development:
- Understanding your data — actually looking at real data comes first
- Making your requirements explicit — writing down what you truly want the model to do
- Getting the model to consistently meet those requirements on new data
The book frames AI evals as the most important and hardest part of building AI products, offering a practical on-ramp past the maze of conflicting advice.
More from coding & agent
- Agents are sessions, not requests: Google explains sizing GKE node pools for agent workloads — rseroter · 2026-11-21
- Agent Scale Lessons: Shared Specs, Adversarial Pairs, Retiring Long-Run Agents — LukeW · 2026-11-21
- Agent sandbox boat.dev launches at $0.036/hr, hiring founding engineer at $260k — samsja19 · 2026-11-21
- One stray print() inside an MCP stdio server corrupts the JSON-RPC stream — Independent-Sir3234 · 2026-11-21
- Microsoft's Web IQ rebuilds search for AI agents to cut token costs — JordiRib1 · 2026-11-21
- ClickUp launches Artifacts: one-shot app building inside your workspace — simcdev · 2026-11-21