NVIDIA says AI-assisted CUDA optimization delivered 2x in a month, vs 5x/year by experts
SumitGup · x · 2026-10-21
SemiAnalysis relayed on-site news from the PyTorch conference: NVIDIA engineers said the speed of optimizing CUDA software with frontier AI models has increased rapidly. Their slide (SemiAnalysis InferenceX and AgentX benchmarks) shows:
- Oct 2025–Aug 2026: expert human optimization delivered 5x in 1 year
- Sep–Oct 2026: AI-assisted optimization delivered 2x in 1 month
- Oct 2026: AI orchestration + MegaKernels added 1.5x in 1 week
Benchmarks cover token throughput per GPU and per MW for DeepSeek V4 Pro and Kimi K3 on GB300 NVL72, showing AI dramatically compressing the iteration cycle of low-level software optimization.
More from Infra
- NVIDIA ships FP4-Local checkpoints of Qwen3.8-Flash-Next and GLM-5.3-Flash for local AI — NVIDIAAI · 2026-10-21
- Rumor: post-hoc GQA→MLA conversion cuts Qwen 27B KV-cache memory by up to 8.4x — ChrisGPT · 2026-10-21
- EAGLE-4 deep dive: up to 5.21× agentic coding speedup with lossless speculative decoding — zhyncs42 · 2026-10-21
- EAGLE-4 speculative decoding hits 961.6 tokens/s on Qwen3.8-27B, code open-sourced — zhyncs42 · 2026-10-21
- Bittensor miners speed up REAX decision model nearly 3× within one evening of competition — airesearch12 · 2026-10-21
- Musk says compute demand is 'ridiculous', floats 6-month payback; analyst questions neocloud financing woes — firstadopter · 2026-10-21