1,300+ graded word-association clues: Google models beat OpenAI, both beat Anthropic
Hardicecream17 · reddit · 2026-11-05
A developer graded 1,300+ clues from a word-association benchmark: one model gives a one-word clue for a 5x5 board, a teammate model guesses, scored on how many of its own words are found before hitting a wrong one. Models played paired games under identical rules and grader.
Average clue grades (95% intervals):
- Gemini 4 Argon 67±3, Gemini 3.8 Flash 64.5±2
- GPT-6.1 Terra 64±4, GPT-6 Astra 63±2, GPT-6.1 Sol 61±2.5
- Claude Opus 5.5 58±2, Sonnet 5 57±4, Fable 5.1 56±3
Wrong-board hit rates: Gemini 0–0.5%, GPT-6 0.8–1.9%, Claude 1.7–6%. The cheap tier with thinking off shows the same gap: GPT-6 Luna beat Claude Haiku 5.5 14–2 head-to-head.
Caveats: single task, partly LLM-based grading, and older paid-API models break the pattern (Claude Opus 4.6 topped that table at 55). The author speculates huge consumer user bases may give ChatGPT/Google models more human language-association data.
More from Models
- SkyPilot Tokens lists Kimi K3.1 Pro day-0: 2.8T params, 1M context, 40% cheaper — skypilot_org · 2026-11-05
- New open-source American LLM pretrained from scratch teased, drawing AI community buzz — annadgoldie · 2026-11-05
- OpenAI October dev roundup: Decisions API, 12-hour caching, 8x faster Codex tokens — OpenAIDevs · 2026-11-05
- Researcher laments the enormous cost of RL used to squash models' quirky behaviors — generativist · 2026-11-05
- Gemini 4 C arbon one-shots 4,000-line demo, users say Google fixed model laziness — theREALmarvin · 2026-11-05
- DeepSeek alone handled 53% of OpenRouter's 27T daily tokens; open-weights labs top 75% — Comfortable-Rock-498 · 2026-11-05