Reddit debate: BERT score vs LLM-as-a-Judge vs fuzzy matching for RAG evaluation
DueBid3298 · reddit · 2026-10-25
A Reddit user opens a discussion on the perennial headache of evaluating RAG and chat agents, asking the community for hands-on experience with specific examples on two questions:
- Which evaluation method do you use for your RAG/chat agent?
- Why did you choose it over the alternatives — with concrete scenarios?
Methods under comparison include BERT-based semantic evaluation, LLM-as-a-Judge, fuzzy string matching, and other mechanisms. The thread collects practitioners' real-world tradeoffs and is a useful reference for agent eval engineering.
More from coding & agent
- Designer vibe-codes BUNNY'S HOUSE, a home automation app built aesthetics-first — enggirlfriend · 2026-10-25
- Dev lists free remote-jobs MCP server on official registry for Claude, Cursor and ChatGPT — angelnikolov · 2026-10-25
- Garry Tan's Gstack trims 1k lines of old model scaffolding as it crosses 137k stars — garrytan · 2026-10-25
- Agents still bad at dependent types: their instincts skew imperative even when writing Lean — hargup13 · 2026-10-25
- LangChain's Harrison Chase teases enterprise answer to OpenClaw, offers early access — hwchase17 · 2026-10-25
- Catalogue of 48 programming languages designed for AI agents maps three competing camps — QuixiAI · 2026-10-25