10-hour GPU kernel optimization benchmark: GPT-6 Astra leads at 63/69, Kimi K3.1 Pro close behind
soumithchintala · x · 2026-11-06
Soumith Chintala shared a GPU kernel optimization benchmark (PMPP-Hard, 69 tasks, 4 independent attempts, 10-hour budget) showing frontier models can grind for hours on kernel optimization:
- GPT-6 Astra tops at 63/69, Kimi K3.1 Pro 60.5, Claude Opus 5.5 60
- Kimi K3 55.75, DeepSeek V4.1 Flash 55.25, Inkling 2 Small 54.25
- GLM 5.3 52.5, Mistral Large 4 50.25, GLM 5.2 42
The curves also show convergence-speed differences: GPT-6 Astra and Kimi K3.1 Pro plateau within 1 hour, while Claude Opus 5.5 starts slow but keeps climbing. He adds their pre-trained base model is extremely competitive.
More from Models
- Thinking Machines: Inkling 2 Small live on Tinker, open weights pending safety tests — clarejtbirch · 2026-11-06
- Small-model customizability called critical as Inkling 2 Small touts SOTA fine-tuning — clarejtbirch · 2026-11-06
- TensorFold 1.2.3 boosts Flash Next decode 21–23%; Apache-2.0 Ripple-65B Preview lands — AIFlow_ML · 2026-11-06
- Researcher alth0u teases that a new model's training run is underway — clarejtbirch · 2026-11-06
- Inkling 2 Small pretrained on 8.4T tokens in 68 hours, natively in NVFP4 — clarejtbirch · 2026-11-06
- Why do Astra and Sol 6.1 perform so well? Multi-agent RL and epistemic vigilance — scaling01 · 2026-11-06