Blind test of 12 models finds Fable 5.1 reads least like AI at 14%, Gemini 3.8 Flash worst at 77%
PawelHuryn · x · 2026-09-07
- Pawel Huryn ran 3 writing tasks × 3 tries per model with no system prompt — 108 texts, 1,647 blind pairwise judgments by four AI judges (Gemini, Grok, Opus, GPT-5.6 Sol), discarding 30% that flipped when order was swapped.
- Results (lower = less AI-sounding): Fable 5.1 14%, Grok 4.6 22%, Opus 5 24%, GPT-6 Astra 30%; GLM-5.3 55%, Kimi K3 56%, Qwen 3.8 Flash 62%; GPT-5.6 Terra 75%, Gemini 3.8 Flash 77%.
- Awkward twist: the benchmark was built with Fable 5.1, which won anyway — scored only by Google, xAI and OpenAI models since judges never score themselves.
- Human check: model LinkedIn posts beat his pre-ChatGPT (2021-22) posts 355-0. Data and texts published on GitHub within an hour.
Related event: Blind Test of 12 Models: Gemini 3.8 Most 'AI-Flavored'(3 posts)→
More from Models
- OpenAI upgrades Decisions API: 400 questions per call, native voice and image support — OpenAIDevs · 2026-10-26
- Fangcun AI, China's first independent frontier AI safety lab, launches with DeepSeek eval report — JacquesThibs · 2026-10-26
- When AI hits an existential crisis: model confuses itself and everyone else — Ok-Delivery-7437 · 2026-10-26
- Kimi K3.1 Pro tops MathArena open models, closing frontier gap to about 6 months — teortaxesTex · 2026-10-26
- Ruff creator Charlie Marsh: frontier models find infinite bugs in any software — charliermarsh · 2026-10-26
- Dev testing one skill across Opus 5.5 and Sol 6.1 finds instruction following 'all over the map' — lucasmeijer · 2026-10-26