Embedding test: RAG ranks the no-refund policy first, showing models still matter
galratner · reddit · 2026-09-25
The author embedded two contradicting refund policies with nomic-embed-text and queried "Can I get my money back if I cancel my wedding?" The no-refund sentence ranked first (0.7260 vs 0.7216), and across five phrasings the pair never differed by more than 0.02. Embeddings encode what a text is about, not whether it says yes or no; vector search also has no concept of "no match" — something is always nearest.
Citing a Stanford/Yale preregistered study of 202 legal queries: Lexis+ AI hallucinated on more than 1 in 6 queries, Westlaw's AI-Assisted Research on 1 in 3, with reasoning errors contributing to 61% of Westlaw's hallucinated answers — while retrieved results often looked relevant.
The author runs a production AI wedding-planner agent (Claude Opus via Microsoft.Extensions.AI, a .NET 10 MCP connector to the booking system, Sonnet fallback on API overload) and argues retrieval decides what the model sees, but weights decide what it does: spotting contradictions, applying policy precedence, admitting ignorance. Asks whether rerankers can reliably separate such contradictions; reproduction code included (C#, TextSearchProvider, SQL Server 2025 vectors, DiskANN read-only gotcha).
More from coding & agent
- Mk1.5 explained: factor world knowledge into tool calls, parallel research via sub-agents — AkshatS07 · 2026-09-25
- Dev open-sources Goated Prompter, a local browser prompt generator recreating a gatekept ComfyUI node — Jolanoff · 2026-09-25
- Solo Dev Automating SMB Workflows for 3 Years: From $1.1K–3.8K/Month to One Contract Matching a Month's Income — Kakachia777 · 2026-09-25
- Solidity Auditor v4 open-sourced with 85% recall on High findings, ported to Vyper — banteg · 2026-09-25
- I Analyzed 246 Repos and 57 Papers on Agent Harnesses — Here's What Actually Works — Marmelab · 2026-09-25
- Cua ships stable Driver for Omarchy, enabling OS-level multi-cursor computer use — TianbaoX · 2026-09-25