Zero evals is fine at first: nail UX, then build evals with domain experts
Al_Grigor · x · 2026-10-30
AlGrigor shares practical advice on evaluating AI apps: starting with zero evals is fine as long as you can build an eval set when needed.
- Figure out UX first — pouring time into evals before validating the product design wastes them.
- When it's eval time, sit down with domain experts — they define what "good" means; learning from the people models are built for has been his most valuable practice.
- He cites Paul Iusztin's article on taking a RAG FAQ bot for DataTalksClub from vibe checks to a shippable 61-case eval suite in 5 steps.
Related event: Engineers: treat custom evals as first-class citizens(2 posts)→
More from coding & agent
- TestSprite auto-generates 35 end-to-end browser tests for an AI-built RSVP app, 34 pass — alexisjross · 2026-10-30
- Reddit: The biggest thing agents gave me this year is sleeping through the night — ChemicalExcellent463 · 2026-10-30
- OpenAI DevDay community edition lands in Stockholm Nov 21 with $150 in credits — paw_lean · 2026-10-30
- Microsoft's Work IQ connects business app context, unveiled at PPCC26 keynote — donasarkar · 2026-10-30
- Open-source ComfyUI light stage: drag lights around your subject for AI video relighting — Holiday-Opposite8514 · 2026-10-30
- Codex turns Apple Photos pets into a Samsung Frame TV art channel — sch · 2026-10-30