GPU Offload in Rust: Portable, Safe, and Fast
Manuel S. Drehwald, Marcelo Domínguez, Kevin Sala, Alán Aspuru-Guzik, Johannes Doerfert
cs.PL
2026-08-14
A GPU offload framework built natively into rustc and the LLVM toolchain lets Rust kernels run on NVIDIA and AMD with compiler-derived data movement, matching hand-optimized CUDA/HIP kernel times on RAJAPerf (whole-suite range: 32% faster to 46% slower).
HPC and scientific computing still run on C, C++, and Fortran with CUDA or HIP. None of these languages have safety boundaries: transfer sizes, buffer ownership, and thread discipline all live in the programmer's head. Rust enforces memory safety and data-race freedom at compile time, yet getting Rust onto GPUs has meant bad trade-offs. rust-gpu targets SPIR-V and lacks general pointers. rust-cuda requires unsafe blocks for every kernel. NVIDIA's cuda-oxide is safe but vendor-locked. Pick two of three: safety, portability, performance.
This paper's bet is to build GPU offloading into upstream rustc and LLVM's Offload infrastructure rather than bolt on another DSL. The team spans the University of Toronto, LLNL, and URJC, with Rust Foundation funding and a champion inside the Rust language team.
The framework offers three interfaces, ordered by how much the compiler manages:
The design leans on Rust's type-system dividends. Mutable references already carry the noalias attribute in LLVM IR, no manual restrict annotations required, which hands the backend an unusually clean optimization baseline.
Kernel safety comes from a partitioning abstraction. Sharing a &mut slice across GPU threads is undefined behavior, and the usual escape is raw pointers everywhere. Here, "which elements may each thread touch" is split out of the kernel: every Region binds a PartitioningStrategy that guarantees disjoint thread regions, so kernel bodies stay in safe Rust. The strategies implement an unsafe trait but are plain Rust, and users can write their own.
Transfer direction is derived from types as well: &T and const T lower to a one-way MapTo, &mut T and mut T to a bidirectional MapToFrom, and scalars up to 64 bits pass by value. What OpenMP makes you spell out in data-mapping pragmas is generated automatically by a MIR-level analysis.
The toolchain compiles in three passes: collect host-side kernel monomorphization metadata, compile device bitcode for the GPU target and package it, then compile the host side and embed the fat binary. The authors rejected cuda-oxide's single-pass design because #[cfg(targetarch)] selects implementations per target, so a single pass can leave x86 inline asm inside device code; translating such target-specific code to GPU IR is, in their judgment, infeasible. The cost is explicit cross-pass communication, handled by a monomorphization metadata query.
Thirteen RAJAPerf kernels were ported to pure Rust and run against RAJA's hand-optimized CUDA/HIP backends on an AMD MI250X and an NVIDIA H100, using a rustc based on LLVM 23.1.0-rc1:
| Dimension | Rust | Baseline (CUDA/HIP) |
| Whole-suite runtime, MI250X | up to 32% faster | down to 43% slower |
| Whole-suite runtime, H100 | up to 11% faster | 44%/46% slower on FIR/LTIMES |
| H100 H2D transfers | 53 transfers, 423 MB | RAJA: 55, 468 MB |
| H100 D2H transfers | 9 transfers, 69 MB | RAJA: 9, 99 MB |
| H100 total transfer time | 46 ms | RAJA: 16 ms |
| Avg registers, RTX 2070 | 33 | RAJA-CUDA: 28 |
Raw kernel times are on par with RAJA; the gaps concentrate in tiny kernels like FIR and LTIMES, which the authors attribute to differing unroll decisions across the three compilers rather than a framework tax. More awkward: Rust moves less data but its transfers take nearly 3x longer (46 ms vs 16 ms), suspected to come from memory-kind and async-transfer differences, unresolved in the paper. One more shot at ergonomics: naive Interface A, transferring once per kernel launch instead of once per benchmark, runs over 400x slower than the explicit interface. The team prototyped prefetching, loop hoisting, and intermediate-transfer elimination in LLVM's OpenMP-opt pass and claims these bring the convenient interface to parity on RAJAPerf; that part is a prototype outside the evaluation tables.
On floating point there is a Rust-specific wrinkle: fast-math is off-limits because its nnan/ninf assumptions can trigger UB in safe code. Experimental algebraic floats keep most of the optimization opportunities without those two assumptions, delivering a 2x speedup on FIR and roughly 20% on three other kernels on an RTX A2000, with no significant gain on MI250X.
This is Rust GPU programming moving from community side-projects to mainline compiler infrastructure. cuda-oxide covers NVIDIA and leaves AMD and Intel users out; this work sits on LLVM Offload, emits native code for NVIDIA and AMD today, and extends to Intel as that target matures upstream. For HPC it offers a compile-time memory-safety path alongside Fortran/C++; for Rust developers it points at writing kernels without unsafe blocks. Both hold only if the work actually lands upstream.
The authors are upfront about several. Two compilation passes mean host and device type layouts and ABIs can diverge; they hit a real case on plain slices, where x8664 and amdgcn lower a slice to two scalars while nvptx64 lowers it to a fixed [i64; 2] array, reconciled by hand. The Rust standard library does not run on GPUs yet. On the evaluation side: the 3x transfer-time loss is explained only as a suspicion; the FIR/LTIMES gap is attributed to unroll decisions without a breakdown; and a 46% worst-case whole-suite deficit is not small for HPC users who need predictable performance. The central claim that the 400x ergonomic gap is erased by compiler optimization rests on prototypes, with no systematic numbers in the paper. This is also a prototype-stage effort: "integrated into upstream rustc" is part design intent, part implementation, and LLVM's Offload split itself is still in progress.