GaussianRenderFormer: Hebrew U team renders 3D Gaussian splats with a pure transformer, no rasterizer, hitting 38.08 dB

HUJI-IL · hf · 2026-11-05

A Hebrew University team presents GaussianRenderFormer, replacing the rasterizer entirely with a neural network for 3D Gaussian Splatting rendering: each Gaussian becomes a token in a view-independent encoder, and a view-dependent decoder maps small patches of camera rays to pixels via attention.

Key facts and results:

Trained on 26,820 Objaverse objects, each pruned to 20,000 Gaussians;
On 300 unseen objects it renders the full, unpruned splats at 38.08 dB PSNR, versus 44.02 dB for rasterizing the same pruned input;
The authors' diagnostics indicate the gap is not generalization but a bottleneck in the view decoder: telling it each Gaussian's projected footprint adds 1.4 dB;
Under-training is not ruled out; code, models and videos are open-sourced.

A notable empirical step in replacing fixed graphics pipelines with learnable components—showing the implicit geometric knowledge in the rasterization pipeline is still hard to learn end-to-end.

Original post →

GaussianRenderFormer: Hebrew U team renders 3D Gaussian splats with a pure transformer, no rasterizer, hitting 38.08 dB

More from Multimodal