9 fake software brands, 18 queries to ChatGPT and Google AI Mode: zero hallucinations
raja314 · reddit · 2026-11-16
A Reddit user fabricated 9 software brand names (unfindable on Google) and asked ChatGPT and Google AI Mode each one's price and founder: 18 answers, zero invented. ChatGPT always said it couldn't verify existence and asked for a website; Google AI Mode said they don't exist and suggested similarly-named real products.
Testing 26 real companies on founding year and HQ: famous 19/20, mid-sized 14/14, lesser-known 14/18 — all 5 misses involved contradicting sources (e.g., Lexalytics HQ: ChatGPT cited LinkedIn saying Amherst vs Wikipedia's Boston), where models picked a side rather than making things up. On 8 SaaS pricing questions: 16/16 matched the headline price on vendor pages, with misses only on monthly-vs-annual toggles.
The author's takeaway: with internet access, both engines chose between conflicting sources instead of fabricating — a different scenario from what public hallucination scoreboards test (document summarization or deep questions without documents), which explains their wildly swinging numbers. Caveats: one day, one location, one correct answer per question.
More from Models
- NVIDIA's Free Nemotron 3.5 Bolt E4B: A 4GB Model That Runs Offline on Almost Any Laptop — alex_verem · 2026-11-16
- Gemini 4 Hydrogen doc analysis tested: one prompt yields report plus comparison table — technextpreneur · 2026-11-16
- Anthropic S-1: Models Older Than 6 Months Earn Under 20% of Revenue — every · 2026-11-16
- OpenAI weekly: ChatGPT Wallet rollout, Odessia acquisition, voice modes retiring Dec 11 — btibor91 · 2026-11-16
- AI-built 367K-parameter LM fits on a floppy disk and runs on a Pentium — RichmanRonald · 2026-11-16
- Inference providers are not hosting Perplexity's decider model despite production demand — bo_wangbo · 2026-11-16