Grok 4.7 medium beats xhigh in independent tests, sparking benchmark-gaming doubts
PawelHuryn · x · 2026-09-22
PawelHuryn's independent tests show Grok 4.7 (medium) consistently beating (high) and even (xhigh): medium scored 30/23/27 vs xhigh 25/30/31/29 and high 19/22/17 (low: 11/15). Grok 4.6 (high) at 23 also beat 4.7 (high) at 19. He is expanding experiments across effort levels but suspects 4.7 wasn't fixed — just optimized for popular benchmarks.
Related event: Independent tests show Grok 4.7 medium reasoning beats xhigh(2 posts)→
More from Models
- Robot build test: GPT-6 Astra stabilizes two-board structure, Claude Fable 5.2 falls short — qinzytech · 2026-11-12
- Zvi dissects Anthropic's latest system card, disputing the autonomy threshold call — Don't Worry About the Vase (Zvi) · 2026-11-12
- Unsloth ships Qwen3.8-27B Dynamic V3.1 GGUFs with 5-10% accuracy gains — danielhanchen · 2026-11-12
- antirez backs low-bit DeepSeek 4.1 Flash over Qwen 27B dense on Apple M5 Macs — antirez · 2026-11-12
- Gemini re-renders images as text when vision is removed, dropping only 6% WSR — zacharynado · 2026-11-12
- Bare-harness coding benchmark: Fable 5.2 best overall, Gemini 4 Carbon best value — Semantic_meaning · 2026-11-12