Score retrieved passages with an LLM: below 0.15, admit ignorance instead of hallucinating
TejasKumar_ · x · 2026-10-02
- The author's personal Q&A bot uses one LLM call to score each of the top 10 retrieved passages: "does this answer the question?"
- Two attack questions score 0.06 and 0.07, while the lowest of 60 legitimate questions is 0.13 — so a 0.15 threshold makes the bot say it hasn't covered a topic instead of making things up.
- It complements the prompt-injection guard in the same thread, forming one cheap anti-hallucination + anti-injection workflow.
More from coding & agent
- Microsoft paper: coding agent optimizing prompts from logs beats GEPA at ~$1.60 — rohanpaul_ai · 2026-10-02
- Stop reaching for the biggest model: a cost-efficient Cursor/Codex setup with GPT-6.1 Sol — chiliraupe · 2026-10-02
- Dot isn't better than Codex or Claude Code — it's a different, undervalued take on OpenClaw/Hermes — gabrielchua · 2026-10-02
- Engineering.com parent Arrowfly launches year-round AI for Engineers initiative amid vibe-CAD era — burhop · 2026-10-02
- Run Fewer Agents: exe.dev argues task management is a band-aid, proposes fast models for human comms — sull · 2026-10-02
- Developer gives an AI agent $1,000 to run a live-streamed hedge fund — kleffew94 · 2026-10-02