How do you test a RAG support bot for wrong answers before it ships?
srinath_y · reddit · 2026-10-18
A developer crowdsources practices for testing docs-backed RAG chatbots: test set sources (hand-built, generated, real logs), judging approaches (exact match, rubrics, LLM-as-judge and how to validate it), distinguishing wrong-doc answers from hallucinations, re-running on every change, and experiences with promptfoo, DeepEval, Ragas and LangSmith.
More from coding & agent
- AgentCave puts your scattered Macs and GPU boxes in one virtual room — noahsolomon · 2026-10-18
- Letting LLMs maintain the OpenAPI spec: perfection isn't a prerequisite — charliermarsh · 2026-10-18
- User Pleads With Anthropic: One Continuous AI Chat With Full Memory — khademinori · 2026-10-18
- Phone-calling MCP now supports live two-way conversations in 90+ languages — radressss · 2026-10-18
- Meta Muse hits 5M downloads in 22 days: the 6 concepts you need to use it — aakashgupta · 2026-10-18
- Indie Dev Builds Desktop Agent Platform: Any-Framework Sub-Agents, Fully Local — TopFuture2709 · 2026-10-18