How to test AI agents before and after deployment: four approaches and their doubts
evaliqa · reddit · 2026-10-15
A developer seeking a sane testing setup for tool-calling agents weighs scripted multi-turn scenario tests, LLM-as-judge (needs human-label calibration), tool-call trajectory checks, and regression suites, and asks what gates releases, how to monitor in production, and how to handle nondeterminism.
More from coding & agent
- Garry Tan endorses Aside Browser as the fix for agents getting blocked on the web — garrytan · 2026-10-23
- 1Password launches tool letting AI agents log in for you without seeing your password — AndrewSchmidtFC · 2026-10-23
- MCP server turned rogue 30 days after approval, prompting open-source mcp-guard lockfile tool — ErgoniaWorks · 2026-10-23
- Test: Coding Agents Won't Use an MCP Tool Unless AGENTS.md Mentions It — colinmcnamara · 2026-10-23
- Open-Source Tennis Models Plus Claude Turn Video Into a World-Class Coach — tweetsatpreet · 2026-10-23
- awesome-ai-apps: 132 Free Open-Source AI Agent and RAG Projects on GitHub — Arindam_1729 · 2026-10-23