Grok 4.7 [xhigh] solves all five KernelBenchMega GPU kernel tasks, reward-hack audited
scaling01 · x · 2026-09-22
elliotarledge ran Grok 4.7 [xhigh] on KernelBenchMega (kernelbench.com) with an RTX PRO 6000 rig: on an unlimited budget every session self-terminated within 100 minutes, all kernels correct, and every number is an isolated regrade after a l trace-level and submission-level reward-hack audit.
The benchmark has five kernel tasks: Kimi-Linear Decode megakernel, Grid + MinGRU (basically a sim), Native Sparse Attention, GLM-5.3's Fused MoE, and MegaQwen Decode. The leaderboard shows most frontier models scoring 0 on most tasks, with a few passing or flagged.
More from coding & agent
- YC-backed Roma turns your to-do list into an agent command center across Gmail and Notion — testingcatalog · 2026-11-12
- Boubou: open-source CLI turns your laptop into a fully local voice AI hub — alexcovo_eth · 2026-11-12
- User automates entire video clipping workflow with Nous Research's Hermes agent — Teknium · 2026-11-12
- Tool-call failure rates jump 14x across providers, charts pinpoint inference stack updates — remilouf · 2026-11-12
- As models improve, writing good evals gets harder and more important — amankhan · 2026-11-12
- Prime agent ships with day-one multi-agent communication, runs in Minecraft unmodified — sarahcat21 · 2026-11-12