Langfuse workshop: full evals loop for high-stakes AI agents, from tracing to CI
AI Engineer · youtube · 2026-10-18
AI Engineer published a hands-on workshop with Langfuse's Doneyli De Jesus and Annabell Schäfer walking through the full AI engineering eval loop using a banking assistant: tracing with PII masking, building datasets, turning domain-expert comments into failure modes, choosing between LLM-as-judge and code evaluators, CI that blocks prompt changes when scores drop, stakeholder dashboards, and working with your coding agent. A solid reference for eval engineering on high-stakes agents.
More from coding & agent
- An Agent Can Remember Perfectly and Still Be Wrong: What Evidence Proves 'Done' After a Context Reset? — jylusdev · 2026-10-18
- Grok Bot vs Claude Code: When Compute Is Elastic, Platform Capabilities Decide — tekbog · 2026-10-18
- A prompt to make your agent audit under-used features across your MCP-connected tools — thejasminejade · 2026-10-18
- DeepSeek V4.1 Flash 1017 matches Opus 5.5 on long-horizon browser use at 19x lower cost — zizhpan · 2026-10-18
- Writer has Claude redesign her Obsidian plugin UI with zero backend changes — evielync · 2026-10-18
- AI agents hunting a $465 train refund invented excuses, leaked medical data, and spammed an MP — The Decoder · 2026-10-18