America.gov Stress Test: 9 of 15 Reporting Questions Correct, Zero Hallucinations
hannotek · reddit · 2026-09-30
The day after GSA launched America.gov, an AI "front door" covering 29,000+ government sites, bureaucracy.news stress-tested it with 15 questions drawn from its own reporting where the true answers were already known.
Results
- 9 correct, 6 substantially correct but incomplete, 0 fabricated answers
- Notably, it declined to invent numbers when it didn't know: "I cannot invent that number."
Weaknesses
- Found the right DHS OIG report but claimed contractor counts weren't available; the report states 1,208 separated CBP contractors kept access and 623 separation records were delayed over 30 days
- Nuclear waste figures accurate but stale
- Flagged a public DOE contract ID as "personal information" and blocked it
- Couldn't verify 10 recommendations in a classified clocks audit, though Oversight.gov lists them
Bigger issue: GSA hasn't published the website inventory, definitions, update frequency, source ranking, models, or change logs. Search isn't a records request — America.gov shows what's posted, not what the government hasn't released; FOIA does that.
More from Safety
- X nears launch of adding Grok to group chats, with encryption caveats — XFreeze · 2026-09-30
- A 'wisdom gradient' for AI alignment: eliciting diverse human values bottom-up — edelwax · 2026-09-30
- Your MCP anonymizer is self-defeating if it takes code as an argument — Aggressive-Course-24 · 2026-09-30
- AI Is a Software Product: Labs Can't Dodge Liability With 'AI Did It' — gerardsans · 2026-09-30
- Poison sample selection swings LLM backdoor attack success from 3% to 80% — chhaviyadav_ · 2026-09-30
- Stuart Russell: the safest AI might be one that doesn't know what we want — JMarty97 · 2026-09-30