GPU QR competition audit: winning kernel ran 107x faster yet failed on new inputs

kfountou · x · 2026-11-18

A new paper argues that as AI coding agents produce numerical software faster than experts can review it, additional evidence of correctness is needed. The authors audited a public GPU QR competition: the winning entry ran 107x faster than the PyTorch/NVIDIA QR baseline, yet all five leading entries — despite passing every accuracy test the competition ran — failed on new inputs from the competition's own generator under the same checker, while the baseline passed all. Heatmaps show 30–66% error rates on rank-deficient, banded, and mixed inputs. The paper proposes VERNA, an audit pipeline (recognize → test against error bounds → certify into a certificate of trust) paired with VSCL machine-checked error bounds, as a framework for trustworthy AI-written numerical kernels.

Original post →

More from Research

Research channel →