How do you actually monitor and evaluate your agents in production? Devs compare LangSmith, Phoenix, Ragas
rcleffke · reddit · 2026-10-22
A Reddit developer asks how teams monitor and evaluate agents in production, where prompt changes or model updates can silently break things. He shares his own rankings site's take — LangSmith leading, Arize Phoenix (20.24.0, Oct 21) as the open-source self-hostable option, Ragas for free CI-runnable RAG metrics — and asks what others trace, whether they use exact-match tests, LLM-as-judge or human review, and if any of it runs pre-deploy.
More from coding & agent
- Free book 'LangGraph in Small Pieces' teaches agents from blocks to production — truckkicker · 2026-10-22
- Cloudflare's AI security-audit skill tops GitHub with 38.5k stars — alex_verem · 2026-10-22
- Roboclaw agent autonomously traces and fixes OpenClaw UI bug, PR merged — heyneighbor · 2026-10-22
- Bruin open-sources LLM inference as a first-class primitive in data pipelines — karakanb · 2026-10-22
- YC's Design Review: Magic Patterns demos MagicOS canvas and dissects why startup sites feel 'vibe coded' — ycombinator · 2026-10-22
- Skills over MCP plus progressive disclosure: fixing context bloat in domain-specific AI systems — sebgehr · 2026-10-22