1,300+ graded word-association clues: Google models beat OpenAI's, both beat Anthropic's
Hardicecream17 · reddit · 2026-11-04
A developer used his Word Spies game (Codenames-style, with live clue grading, embedding maps and win odds) to benchmark models: one model gives a one-word clue, a teammate model guesses, across 1,300+ graded clues.
Key results (average clue grade, 95% interval):
- Gemini 4 Argon: 67±3; Gemini 3.8 Flash: 64.5±2
- GPT-6.1 Terra: 64±4; GPT-6 Astra: 63±2; GPT-6.1 Sol: 61±2.5
- Claude Opus 5.5: 58±2; Sonnet 5: 57±4; Fable 5.1: 56±3
Wrong-word/assassin hit rates: Gemini 0-0.5%, GPT-6 0.8-1.9%, Claude 1.7-6%. Cheap tier with thinking off shows the same gap: GPT-6 Luna beat Claude Haiku 5.5 14-2 over 16 games.
Caveats: single task, partly LLM-graded, and older paid-API models break the pattern (Claude Opus 4.6 topped that table at 55). The author speculates large consumer user bases give Google/OpenAI more human-language data.
Related event: Google Beats OpenAI in 1300+ Clue Word Association Test, Anthropic Lags(2 posts)→
More from Models
- eliebakouch: This Is Also a Great Way to Distill Chinese OSS Models — eliebakouch · 2026-11-04
- EdgeRock-4B hits 82.9% on AIME with fixed-size memory, runs 512 chats on one GPU — eyishazyer · 2026-11-04
- Mistral-style open-model labs adding closed APIs signal client pressure, says researcher — eliebakouch · 2026-11-04
- Blogger Shows Overseas AI Digital Workforce: GPT Backend, Claude Edits, Gemini Writes — lxfater · 2026-11-04
- India's CERT-In gets access to Anthropic's Claude to test cybersecurity vulnerabilities — kingslyj · 2026-11-04
- Claude Opus 5.5 flags its own proudest sentence as untrustworthy, user observes — RileyRalmuto · 2026-11-04