User Test Finds Gemini 3.8 Flash Hallucinates on Korean Webtoon Quiz, Claude and ChatGPT Pass
DK1530 · reddit · 2026-09-13
A heavy Gemini user ran a small hallucination test with five Korean-language prompts about the Korean webtoon GO-SU (with minor intentional misspellings) across three models:
- Gemini 3.8 Flash (app version, no extended thinking): started hallucinating from question 2.
- Claude 5 Sonnet (free tier): answered all five correctly.
- ChatGPT (free tier, model name hidden): answered all five correctly, with one minor hallucinated detail in context.
Disappointed, the author ran two more tests: Google AI Mode and Gemini 3.8 Flash inside Google AI Studio both answered all five correctly. The author concludes the app version of Gemini 3.8 Flash is notably worse, and stresses this was just a casual test for fun.
More from Models
- Bindu Reddy: AI slowdown talk hands Google a catch-up window, Gemini 4.0 may be strong and dirt cheap — bindureddy · 2026-09-13
- Researcher floats Pliny, Janus, and Lisan as third-party frontier model auditors — DimitrisPapail · 2026-09-13
- Search vet Doug Turnbull: babysitting sloppy frontier models vs. 'superhuman' hype — lateinteraction · 2026-09-13
- Unverified rumor claims DeepMind has reached recursive self-improvement internally — MrSnowden · 2026-09-13
- Moonshot denies CEO arrest rumor but stays silent on distillation claims — pstAsiatech · 2026-09-13
- Report: Musk and Altman back Amodei's call to slow AI development — AIFlow_ML · 2026-09-13