Grok 4.5 Benchmark Scores Impacted by Test Set Leak
SkyLi0n · x · 2026-07-10
Reports indicate Grok 4.5 holds an advantage on the CursorBench benchmark because its training data accidentally included an early snapshot of the Cursor codebase. The team noted the exact impact on scores is unclear, but the data has been removed from future model training.
Related event: Grok 4.5 Benchmarks Strong but Faces Data Controversy(6 posts)→
More from Models
- Astra Scores 83% on GauntletBench, First Computer-Use Agent to Beat Human Baseline — ducha_aiki · 2026-09-11
- Kimi K2.8 Preview rolls out: near-K3 coding performance, 1M context for all tiers — teortaxesTex · 2026-09-11
- Looking for a classifier of software engineering task shapes to pick models per task — StewartalsopIII · 2026-09-11
- DeepSeek V4 Pro API to continue after Sept 2026, billing unchanged — teortaxesTex · 2026-09-11
- DeepSeek V4.1 Flash Hits 98% of GPT-6 Astra's Score at 1.4% of the Cost in Third-Party Benchmark — ayushtweetshere · 2026-09-11
- TheZvi Polls: Has Your Coding Model Choice Changed Since Fable 5.1 and Astra? — TheZvi · 2026-09-11