When 3D Gaussian Splatting Recovers Real Surfaces
Songhe Wang, David Johnathan Miller
ECCV 2026
cs.LG, cs.CV
2026-08-31
Parallax turns geometric error into high-frequency view color. Excess SH allows opaque billboards: 89% of synthetic shapes drift; ten real datasets stay on-surface at SH=24.
3D Gaussian Splatting is no longer just a renderer. Pipelines now carve meshes from learned Gaussians and hang contact or editing on top. The standard 3DGS objective never promised surface recovery: every training view can match while Gaussian centers drift off the true surface. The geometric question is when those Gaussians are actually surface-consistent, rather than buying pixels with view-dependent color.
Volumetric alpha blending entangles visibility, occlusion, and translucency. To talk about geometry alone, rendering is reduced to a first-hit abstraction: on an opaque surface, a ray's color is the appearance function at the first intersection. A pilot on trained 3DGS finds that for 92.6% of foreground rays, the front three Gaussians carry more than 90.2% of compositing mass. That motivates the abstraction in mostly opaque scenes; it does not say 3DGS is first-hit in general.
Appearance is spherical harmonics up to degree L, the angular capacity. On the true surface the best-achievable error A(L) comes only from a smooth specular residual; diffuse texture costs zero angular bandwidth. A misaligned surface must replay background spatial texture as an angular signal through parallax, with induced frequency about spatial frequency times depth error. Below that frequency, wrong geometry carries a floor on error. Above it, an opaque billboard in empty space can memorize the parallax texture with high-order SH and still match every training image.
The intermediate L values are an identifiability window: the true surface is strictly easier to fit. Experiments vary only SH degree, score geometry by normalized Chamfer distance from Gaussian centers to ground truth, and score appearance by PSNR. Opaque billboard means near-best PSNR with badly degraded nCD. The synthetic set has 100 shapes with tunable complexity. Real tests use ten datasets with geometry, including DTU, ScanNet, and Tanks and Temples.
Controlled synthetics match the prediction. In Figure 1, a glinty object at SH=3 reaches 34.9 dB PSNR and 0.03 nCD, on the surface; at SH=24, PSNR is 35.2 dB while nCD jumps to 6.28 and the Gaussians collapse to an off-surface opaque sheet. About 89% of shapes increase nCD as degree grows, while mean PSNR rises slightly. Roughly 53% burst between degrees 6 and 12 and are labeled opaque billboards; about 42% worsen almost linearly and cross the threshold only at the highest degrees. A remaining subset stays within 2× best nCD even at SH=24, the safe regime where spatial texture pushes fake frequency beyond the tested capacity.
On the ten real datasets the same billboard rule never fires. For every dataset, nCD at SH=24 is at most about 1.2× the SH=3 value. The reported mean moves from nCD 0.0220 and PSNR 29.6 to nCD 0.0254 and PSNR 30.2. Under standard multi-view capture, spatial texture is rich enough that L=24 still sits inside the identifiability window.
Raising SH degree fits sharper specularities and also opens a back door for wrong geometry. If Gaussians are going to be used as geometry, angular capacity is a resource to schedule: pin primitives to the surface at low L, then relax. That is also why 2D Gaussians, SDF coupling, and normal or depth regularizers help when appearance capacity is large: they cut off or penalize off-surface solutions. The mechanism applies to any proxy surface with a bandlimited SH radiance head, not only 3DGS.
For common real capture protocols the paper is closer to reassurance than alarm. Under their criterion, high SH does not shove reconstructions off the surface. The failure is forced in synthetic, weakly textured settings, and would be more concerning if someone pushed angular capacity past the L=24 they tested.
The guarantees need mostly opaque scenes, smooth view-dependent residuals, and a diffuse texture that the residual cannot cancel. Mirrors, transparency, strong inter-reflection, exposure, and tone mapping sit outside the assumptions. First-hit is an analytic device, not a faithful 3DGS renderer. Billboard labels use relative nCD and a PSNR band defined in the supplement. Dataset names in the HTML table were scrambled, so per-dataset rows are not quoted here; the 1.2× bound and the mean row are taken from the prose. Geometry-aware 3DGS variants are explicitly out of scope for the same failure claim. Sparse-view capture is untested.