141,750 Explanations Tested: LLM Denial Reasons Are True but Fixes Still Fail
alex_verem · x · 2026-11-18
The arXiv paper "Right Reason, Wrong Advice" systematically evaluates LLM-written denial explanations, introducing recourse validity: when multiple rules independently deny a request, citing one true reason leaves the applicant denied after fixing it.
Setup: 20 LLMs tested across 7 rule systems ported from real ones (SNAP, public housing, federal student aid, access control), yielding 141,750 explanations.
Key findings
- In 6 of 7 systems, naming a single correct reason almost never gave a working fix when multiple rules applied.
- Standard correctness checks were near coin-flip; LLM judges rated 84–89% of explanations "correct" while missing the failure.
- Asking a strong judge "would acting on this get the request approved?" separated working from broken explanations almost perfectly.
- Small local open models gave a working fix only 23–70% of the time; paid Anthropic and OpenAI models reached 92–99%, but averaged only 84% with four-plus independent rules.
- One added prompt sentence — "name every rule behind the denial" — lifted every open model below 90% by 14–61 points.
Context: US states are testing AI-drafted food-aid denial notices; the author argues such notices should be judged solely by whether acting on them gets people approved.
Related event: Study: LLM Denial Explanations Are Right but Advice Fails(2 posts)→
More from Safety
- Are system failures really at record highs? Data says high but not unprecedented — emollick · 2026-11-18
- Microsoft Denies AI Caused the Outlook Outage: July Fix Bug Plus Heavy Load to Blame — Optimum1997 · 2026-11-18
- AI-powered attacks land in four days: your patch window is no longer a decade — ccerrato147 · 2026-11-18
- Chinese phishing service Jieyou adds AI bot: fake login pages via plain language — ccerrato147 · 2026-11-18
- Google caught a contractor building a 100,000-fake-account platform run by AI — ccerrato147 · 2026-11-18
- Stolen AI access resold via 'LLM Mixers' at $0.10–$25 per million tokens — ccerrato147 · 2026-11-18