Grok 4.7 flops in 100 multi-agent coding evals despite insightful solutions
teortaxesTex · x · 2026-09-23
In 100 multi-agent coding evaluations, Grok 4.7's working solutions were often more insightful than comparable frontier models, but it wasted turns and whole submissions on syntax and runtime errors and responded much slower than Grok 4.6 — overall a flop, per the tester. Third-party and unverified.
More from Models
- Theory: Typos Disrupt LLM Tokenization, Which May Explain Why AI Text Has a Distinct Smell — menhguin · 2026-11-11
- LMArena co-founder explains AI evaluation methodology and policy in new podcast — MattPerault · 2026-11-11
- Runway open-sources Praxis-1, turning video pretraining into real robot control — runwayml · 2026-11-11
- DeepMind launches AlphaGenome 2, spotting nearly 2x more gene-regulating mutations — GoogleDeepMind · 2026-11-11
- MazeBench: Fable 5.2 scores 15%, near-frontier at half price; Gemini 4 Carbon 11% — patience_cave · 2026-11-11
- ERNIE 5.5 Flash Preview enters LMArena Battle Mode for crowdsourced ranking — arena · 2026-11-11