3D Gaussian Accelerated Ray Tracing: Fast training through particle-based backward propagation
Laurent Vit, Oliver Batchelor, Richard Green
cs.GR, cs.CV
2026-08-18
3DGART reorganizes Gaussian ray-tracing backprop around primitives instead of pixels, training about 4x faster than 3DGRT on Mip-NeRF 360 with better quality.
3D Gaussian Splatting renders fast by projecting Gaussian primitives onto the screen and compositing them in tiles, but that screen-space shortcut struggles with view-dependent ordering, reflections, refractions, and shadows. Gaussian ray tracing fixes the visual quality problem by tracing actual rays against each primitive instead of projecting them, but training such a model is slow. This paper's diagnosis is specific: the slowdown is not ray traversal itself. It is the backward pass. Existing implementations assign one GPU thread per pixel, and since many pixels can see the same Gaussian, those threads all try to write gradients into the same primitive parameters at once. That forces heavy use of atomic operations and serializes threads that should be running in parallel.
The authors built 3DGART, which restructures the backward pass around primitives instead of pixels, in three steps. First, for every Gaussian they compute a conservative, perspective-correct screen-space bounding box (AABB) that identifies which pixels the primitive could possibly touch. Second, they use that bound to pre-allocate a compact buffer, called the LT buffer, that stores the accumulated color and transmittance values needed for backpropagation, laid out primitive-first, then tile, then pixel. Third, they build a tile-primitive mapping so that during the backward pass, one thread is responsible for one primitive within one tile: it walks the pixels that primitive covers, accumulates gradients in registers, and only issues a single atomicAdd at the end. The old pattern, many pixel-threads scattering writes into one primitive's memory, becomes one thread gathering reads across many pixels. Tiles are 8x8, a size the authors picked as a balance between load balancing, aggregation overhead, and shared-memory pressure. They also found that third-order spherical harmonics account for 48 of 59 atomic operations, which motivated a hybrid mode: color gradients use the primitive-centric path, everything else stays pixel-centric, trading some speed for lower memory use.
All experiments ran on a single RTX 4090. In a raw speed comparison with no ray-intersection cap, kernel size k=2, and alphamin=1/255: at 1.00M primitives, per-pixel backward took 44m08s versus 12m35s for the primitive-centric version, a 3.51x speedup; at 2.00M primitives, 51m13s dropped to 16m32s, a 3.10x speedup. The gain shrinks as primitive count grows, because each primitive then covers fewer pixels and contention was never that severe to begin with. A component-level breakdown shows the backward pass eating 81-86% of per-iteration time in the pixel-centric baseline; with tile-primitive backward, that same pass runs 7.3x faster at 1.0M primitives and 6.5x faster at 2.0M, landing close to forward-pass runtime. The hybrid mode cuts LT buffer memory to a quarter but runs about 1.5x slower than the full primitive approach.
Against 3DGRT, Nvidia's ray-traced Gaussian baseline, on Mip-NeRF 360: at 2.00M primitives 3DGART hits PSNR 27.51 vs 27.11, SSIM 0.820 vs 0.809, LPIPS 0.211 vs 0.214, and trains in 16m32s versus 62m21s, roughly 3.8x faster and ahead on every quality metric, close to the paper's headline "about 4x" claim. At 1.00M primitives, though, 3DGART's LPIPS is 0.239, worse than 3DGRT's 0.214, with only PSNR (27.31 vs 27.11) ahead. The quality-improvement claim holds cleanly at the 2M-primitive setting, not at 1M.
Ray tracing has always had a case for Gaussian rendering: correct visibility ordering, reflections, refractions, shadows. What kept it out of practice was training cost. This paper doesn't touch the forward rendering model at all. It just reorganizes how gradients move through memory during backpropagation, and that alone brings training time from impractical to roughly the same ballpark as rasterization (16m32s versus 20m19s for 3DGS). The slow training of ray-traced Gaussians looks like a systems-engineering problem more than an algorithmic one. The memory-layout trick here, reorganizing scatter into gather without changing the underlying math, is a pattern worth knowing for anyone building GPU training pipelines with contended atomic writes.
The authors state two limits directly. Their perspective-correct projection only supports standard pinhole cameras; fisheye and other strongly non-linear projections are not supported. The LT buffer adds VRAM overhead that grows with resolution, and while the hybrid mode cuts that overhead by 4x, it gives back some of the speed gain, and the authors admit they have not found a solution that avoids both costs.
Table 2 also contains a detail the paper doesn't foreground: at 1.00M primitives, 3DGART's LPIPS trails 3DGRT's, which means the quality edge is not stable once the primitive budget is matched to the competing method. The "faster and better" headline leans on the 2.00M-primitive setting. The two speedup numbers reported also measure different things: the 3-3.5x figure is a controlled comparison of per-pixel versus primitive-centric backward within the same architecture, while the roughly 4x figure against 3DGRT mixes in differences in primitive count, Gaussian kernel shape, and alphamin cutoff. It isn't a clean single-variable comparison. All benchmarks run on one RTX 4090; there's no evidence yet that the speedup holds across other GPU architectures.