Record your judgment process 3 times, let AI build business evals from it
alliekmiller · x · 2026-11-18
Allie K. Miller shares a workflow for building evals on non-coding business tasks (billing, doc filing, website design): the best training/eval data is your own judgment process, not just outputs.
Her recipe:
- Record a screen capture of your entire edit with voiceover
- Narrate the why behind each edit and weigh options out loud
- Provide the finished version to the AI as reference
- Repeat 3 times, then have AI generate an eval set from those three judgment-process + output samples
More from coding & agent
- Founder builds real-time voice support agent with Claude and Agora's AI Developer Toolkit — heyshrutimishra · 2026-11-18
- Audio EQ and compression inside Antigravity: Gemini 4 builds in-chat FFmpeg console — alexanderchen · 2026-11-18
- Auto-approving PRs: Qodo's rules-based workflow handles the coding-agent PR bottleneck — omarsar0 · 2026-11-18
- Dev influencer slams "AI dreaming" claims: it's just a random semantic search DB — gerardsans · 2026-11-18
- A step-by-step playbook for making your enterprise AI-native — alex_verem · 2026-11-18
- The AI coding gap: Limestone ships 98% AI-written code while enterprises lag — alex_verem · 2026-11-18