1,300+ graded word-association clues: Google models beat OpenAI's, both beat Anthropic's
Hardicecream17 · reddit · 2026-11-04
A developer built a Codenames-style benchmark where one model gives a one-word clue and a teammate model guesses, grading 1,300+ clues on how many of its own words a guesser finds before hitting a wrong one.
Key results (average clue grade, 95% interval):
- Gemini 4 Argon: 67±3; Gemini 3.8 Flash: 64.5±2
- GPT-6.1 Terra: 64±4; GPT-6 Astra: 63±2; GPT-6.1 Sol: 61±2.5
- Claude Opus 5.5: 58±2; Sonnet 5: 57±4; Fable 5.1: 56±3
Wrong-word/assassin hit rates: Gemini 0-0.5%, GPT-6 0.8-1.9%, Claude 1.7-6%. The cheap tier shows the same gap: GPT-6 Luna beat Claude Haiku 5.5 14-2 over 16 games with thinking off.
Caveats: single task, grader partly LLM-based, and older paid-API models break the pattern (Claude Opus 4.6 topped that table at 55). The author speculates ChatGPT's and Google's huge consumer user bases may give them richer human-language data.
Related event: Google Beats OpenAI in 1300+ Clue Word Association Test, Anthropic Lags(2 posts)→
More from Models
- How LLM inference actually works: from 140GB weights to GPUs and model serving, explained as a story — _jaydeepkarale · 2026-11-04
- $100 of GPU time edits model weights to cut thinking tokens by 40% — jarrodwatts · 2026-11-04
- Sarvam open-sources Prism, a 44ms decision model that skips text entirely — heyshrutimishra · 2026-11-04
- Frontier models' teaching ability is 'off the charts', says dev: your 'I know kung fu' moment — shauseth · 2026-11-04
- kalomaze: crude diffusion autoencoders may lose low-frequency info at the latent stage — kalomaze · 2026-11-04
- Dev complains frontier models can't do light-touch English editing — A_K_Nain · 2026-11-04