60.33% of 300 LLM GPU kernel issues are compiler, build, or runtime

An Empirical Study of Issues in GPU Kernel Libraries for LLM Systems

Yanjie Gao, Jiayi Bi, Haoxiang Lin, Ling Feng

cs.AI

2026-11-06

A manual review of 300 closed issues across seven GPU kernel libraries finds 60.33% of root causes in the compiler, build, or runtime; software incompatibility alone is 25.33%.

What problem this solves

Most of the runtime in LLM training and inference is attention, matrix multiplication, and quantization. The fast implementations sit in seven GPU kernel libraries: FlashAttention, FlashInfer, Triton, CUTLASS, DeepGEMM, xFormers, and bitsandbytes. A GPU kernel is the device code that actually runs on the card. PyTorch and vLLM own the model graph and the serving loop. Which kernel launches depends on tensor shape, layout, dtype, CUDA version, and GPU architecture at once.

One kernel is reused across models and requests, so a fault travels downstream. It does not have to raise. Generated tokens can be silently wrong, a long job can hang, or a bad package and CUDA pair can fail deployment before the kernel starts. The trigger is often narrow. Change the head dimension or a driver build and the symptom disappears.

Defect taxonomies already exist for deep-learning compilers, frameworks, and training systems. Kernel libraries also have to hold layout contracts, low-precision rounding, and code generation, then dispatch one specialized implementation for the GPU in front of them. That layer had no comparable count of symptoms and root causes.

Method

Microsoft Research Asia and Tsinghua University pulled every GitHub issue from the seven official repositories between 26 December 2015 and 10 July 2026, 9,099 reports in total. They kept closed reports with a concrete failure, dropped questions, feature requests, and documentation, and had 3,181 candidates left. A sample stratified by repository then took 300. If a report had only a symptom, conflicting explanations, or no identifiable cause, it was replaced by the same-repo issue closest in creation time.

Two annotators labeled independently. A mismatch on either the symptom or the root cause counted as a disagreement. Cohen's kappa was 0.87. Each issue got one primary symptom and one root cause. Symptoms are what the reporter could see. Root causes follow the maintainer's diagnosis and the landed fix. Fifteen root causes fall into three dimensions: Kernel Library, Compiler and Build System, and Runtime Environment.

The sample is bitsandbytes 79, Triton 77, CUTLASS 56, FlashInfer 37, FlashAttention 28, xFormers 19, and DeepGEMM 4. DeepGEMM had only 180 raw issues, so four final reports reflect a young repository. Megatron-LM, vLLM, and SGLang as whole systems were out of scope, because their trackers also mix in model loading and distributed scheduling.

Results

More than half of the failures appear before the device runs.

SymptomCountShare
Deployment Failure8227.33%
Build Failure7324.33%
Crash6321.00%
Incorrect Functionality5317.67%
Performance Degradation237.67%
Hang62.00%

Deployment plus build is 155 reports, 51.67%. Deployment stops at install, dependencies, packaging, or import. Build means source or generated code never became an executable. Crash, hang, and incorrect functionality together are 122 reports, 40.67%: the op dies mid-flight, stops making progress, or finishes with the wrong semantics. Hangs are only 6 reports, and they stall long jobs.

Kernel Library accounts for 119 reports (39.67%), Compiler and Build System for 81 (27.00%), and Runtime Environment for 100 (33.33%). The last two sum to 60.33%. The largest single category is software incompatibility, 76 reports (25.33%), covering drivers, CUDA or ROCm, frameworks, operating systems, and ABIs. Next are incorrect compiler transformations, 46 (15.33%), build misconfiguration, 35 (11.67%), API misuse, 26 (8.67%), hardware incompatibility, 24 (8.00%), and code logic errors, 22 (7.33%). Performance bugs add another 17 (5.67%). Incorrect numerical computation is 8 (2.67%), bad indexing 4, and state mismanagement 2.

The contracts are implicit, so a supported shape can still be wrong. On H100, FlashAttention throughput fell from about 370 TFLOPS to 219 when the head dimension went from 128 to 132. Implementations pad awkward sizes out to multiples of 32, 64, or 128 so they line up with tensor cores, and the extra math spends the throughput. xFormers stored the remaining valid key columns in a signed 16-bit integer. Past sequence length 32768 that bound overflows, attention returns NaN, and a downstream VAE decode can come out as a black image. A FlashAttention block-sparse path matched the sparse reference on the forward and then computed gradients as if attention had been dense, because the enabled blocks were not kept for the backward. A Triton mask kernel treated values that change between calls as constexpr, so each call JIT-compiled a new specialization: about 0.7 ms of compile against about 12 microseconds of kernel, enough to stall the CPU on an SGLang serving path. A CUTLASS sparse GEMM accepted FP32 inputs and ran them as TF32, whose shorter mantissa disagreed with a dense FP32 reference until the reference used TF32 semantics too. bitsandbytes installed on a V100 and then failed INT8 matmul, because that path needs INT8 tensor cores the V100 does not have. A CUTLASS Blackwell 2-SM example computed its launch grid from the MMA tile rather than the smaller per-CTA tile and got 4×4 instead of 8×4 on a 1024×1024 output, so part of the result was never computed. Of 82 deployment failures, 47 come from bitsandbytes.

Why it matters

When wiring these libraries into training or serving, check the stack before reading the kernel formula. About 60% of root causes are compiler, build, or runtime, and the biggest slice is a version combination. Bumping one component does not make the new combination supported. One report is a direct comparison: the host had CUDA 11.6, the installed PyTorch had been built against CUDA 11.3, and the same input failed until the reporter switched to a CUDA 11.6 PyTorch build.

Five kinds of tests already show up in the discussions. Record hardware and software versions. Compare the optimized kernel with a reference at the same precision and a stated tolerance. Move one dimension, alignment, or length relationship at a time. Use NVIDIA Compute Sanitizer for illegal accesses and synchronization. Split framework time, compile time, and kernel time. The Triton case is why the split matters. Without it, compile overhead gets recorded as a slow GPU.

A differential test can point at the wrong side. In one FlashAttention report the gradients disagreed with an FP16 reference and agreed with the same math in FP32. The gap was rounding in the reference. Boundary sweeps are just as specific. Head dimensions 8 and 16 failed and 32 passed, which isolated a bad shared-memory calculation. Unequal query and key lengths were required to expose some mask bugs. A kernel that launches is not a kernel whose numbers are right.

The compatibility checker, numerical debugger, pre-launch contracts, and record-and-replay support sketched at the end are research directions. None of them is a tool you can install. There is no faster attention in this paper. The usable output is a priority order: installation, build, and code generation come before another pile of kernel unit tests on common shapes.

Limitations

The sample keeps closed issues whose cause could be pinned down. Ambiguous reports were replaced. That favors well-documented cases. Failures nobody followed up are missing, and so are debugging steps that never entered the thread. The claim that current tests often miss low-precision numerical bugs and hardware-specific execution failures is read off the holes in these reports. Nobody measured recall of an existing test suite.

All seven libraries are open source. Proprietary kernels are absent. So are scheduler and distributed-runtime failures inside Megatron-LM, vLLM, and SGLang. Similar stacks make it plausible that the mechanisms travel: Python or C++ down to CUDA or Triton, the same operators, and the same framework, compiler, driver, and GPU environment. The 15-way mix was not remeasured there. DeepGEMM contributes 4 issues. No repository-level conclusion should be read off it.

API misuse, 26 reports, counts shape, type, and lifetime mismatches at the boundary between wrappers, frameworks, and kernels. Folding that bucket into Kernel Library means the 39.67% mixes both sides of the interface. Code logic errors alone are 7.33%. A kappa of 0.87 says the two annotators were stable. It does not say each issue had only one defensible root cause.

Terms

Source

What people are saying

Related papers

All paper explainers