3500-word guide: building agent evals from real LangGraph traces
strickvl · x · 2026-11-21
HF engineer Hazem Tahir (@htahir111) published a 3,500-word article on building evals for a real working agent.
- Case study: A LangGraph-based expense review agent, using its actual traces as the raw material.
- Approach: Start from the agent's first trace and iteratively arrive at a trustworthy evaluator—reading traces, extracting eval dimensions from failure modes, and codifying human judgment into repeatable evals.
- Series: The article kicks off a planned series to be rolled out over coming days.
A rare hands-on, trace-first agent eval engineering walkthrough rather than generic methodology talk.
More from coding & agent
- Solo dev runs agent swarm in Antigravity to automate SEO analysis — gaganghotra_ · 2026-11-21
- Stanford CS329Z Lecture 9 Released: Optimizing AI Agents via Prompts vs. Weights — stanfordnlp · 2026-11-21
- Stanford CS329Z Lecture 6: AI Agent Design Patterns — ReAct, CodeAct & More — stanfordnlp · 2026-11-21
- 'Fix the Prompt Before You Fine-Tune': Stanford CS329Z's Agent Optimization Roadmap — stanfordnlp · 2026-11-21
- OpenAI agents now finish 46% of 2-4 hour tasks autonomously, up from 23% — AIFlow_ML · 2026-11-21
- Connecting SoduWork to an OpenAI-Compatible Gateway: Config and the 180-Second Stall — JeremyCMorgan · 2026-11-21