Ask Reddit: How do you evaluate production AI agents — mock tools or real calls?
Mountain_Edge1061 · reddit · 2026-10-18
A Reddit poster asks developers deploying AI agents in production (LangGraph etc.) how they run evaluations: do you mock tool/database calls, and do you write traces via Langfuse and evaluate from them, or check the in-memory list of called tools directly? The goal is guarding against regressions when prompts change.
More from coding & agent
- 7-item checklist to stop your agents and automations from failing silently — shannonpieter · 2026-10-18
- Weekly puppet sketch show written, directed and animated entirely by Claude agents — Slackluster · 2026-10-18
- AgentCave puts your scattered Macs and GPU boxes in one virtual room — noahsolomon · 2026-10-18
- Letting LLMs maintain the OpenAPI spec: perfection isn't a prerequisite — charliermarsh · 2026-10-18
- User Pleads With Anthropic: One Continuous AI Chat With Full Memory — khademinori · 2026-10-18
- Phone-calling MCP now supports live two-way conversations in 90+ languages — radressss · 2026-10-18