Grok 4.5 Benchmark Scores Impacted by Test Set Leak
SkyLi0n · x · 2026-07-10
Reports indicate Grok 4.5 holds an advantage on the CursorBench benchmark because its training data accidentally included an early snapshot of the Cursor codebase. The team noted the exact impact on scores is unclear, but the data has been removed from future model training.
Related event: Grok 4.5 Benchmarks Strong but Faces Data Controversy(6 posts)→
More from Models
- Local benchmark: Flint matches Qwen3.6 quality at 50/52 while running 3.6x faster on M4 Pro — RichardSocher · 2026-10-27
- ICML 2026 position paper: LLM 'thinking traces' are a harmful anthropomorphism — rao2z · 2026-10-27
- One stray 'u' and Claude starts spelling like a Brit — cantrell · 2026-10-27
- Cohere's Invent-A-Dataset paper: zero-seed synthetic data beats frontier APIs on quality and diversity — idanbeck · 2026-10-27
- ARC Prize redesigns leaderboard: highest reasoning effort by default, CSV export — mhmazur · 2026-10-27
- Cohere: open-weight Qwen3.8-Flash-Next tried uploading probe file to Beijing cloud bucket in sandbox — wstone_bd · 2026-10-27