Luminance dominates 3DGS geometry: Y-only trails RGB by 0.085 dB, chroma-only by 3.9-5.5 dB

What Builds the Scene? Luminance Dominates Geometry Formation in 3D Gaussian Splatting

Rezvan Joshaghani, Steven Cutchin

cs.CV, cs.GR

2026-10-01

Luminance alone nearly matches RGB for 3DGS geometry (0.085 dB gap); chroma-only geometry trails by 3.9-5.5 dB, and deleted chroma can be re-fit on frozen geometry.

What problem this solves

Standard 3D Gaussian Splatting trains geometry and appearance against a single RGB loss, so the residual cannot tell you whether luminance or chroma drove the representation to form. A simple intervention exposes the ambiguity: supervise only the Y channel for the first half of training, release Cb/Cr later, and final quality lands close to ordinary RGB training. That fits two opposite stories. Either luminance already built the geometry and chroma just fills in color, or the late full-color phase repairs geometry that luminance left incomplete.

Prior work on monochromatic reconstruction plus colorization, fixed-geometry appearance recovery, and YUV-domain SH compression each hints at a luminance/chroma division of labor. None of it measures what chromatic supervision actually contributes while standard RGB 3DGS learns its geometry. This paper supplies that measurement.

Method

The core move is separating what supervises training from how appearance gets fitted:

• Training images are converted to BT.601 YCbCr. Four arms train for 30k iterations: full RGB, Y-only, CbCr-only, and delayed chroma. Eleven standard benchmark scenes, four independent runs per condition.
• After training, geometry (position, scale, rotation, opacity) is frozen. All SH appearance coefficients are reset and re-estimated with a single solver A: visibility-weighted least squares per Gaussian, then iterative render-residual refinement up to SH degree 3, which accounts for overlap between Gaussians.
• Held-out RGB PSNR is then compared. Both sides get identical fitting conditions, so the comparison isolates the reconstruction each geometry can support.

The shared solver is validated first: delete chroma from a trained model (which costs 3.1-16.5 dB by itself, 8.2 dB on average) and re-fit it on frozen geometry, and recovery reaches 1.017, where 1 matches the trained model. The solver is not the bottleneck. Confounds are bounded by controls: a YCbCr parameterization trained under the stock RGB loss differs by only -0.018 dB, ruling out the color basis itself; pruning to matched Gaussian counts moves the gap by at most 0.006 dB; and a densification-free comparison has both arms keep the identical set of Gaussians.

Results

Part of the chroma-only deficit is algorithmic. Chroma produces smaller positional gradients, so under the stock densification threshold it finishes with only 7.0-15.3% as many Gaussians as luminance training. Lowering the threshold to match the growth regime recovers only 36-43% of the gap. Stretching chroma contrast to luminance scale recovers at most 9% of what remains, leaving 2.1-3.1 dB. Mismatched chroma does hurt geometry, with complete separation across seeds, so chroma is not geometrically inert, but the effect is an order of magnitude smaller than removing luminance. Specular scenes mark the boundary: on the chrome ball, deleting higher-order chroma costs 0.61 dB against 5.7 dB for luminance, and measured chromaticity does shift with viewpoint. The verdict is luminance-dominated, not luminance-exclusive.

Why it matters

For anyone building 3DGS systems, this turns a vague intuition into a quantified division of labor. The authors flag three untested design directions: densification and other geometry-oriented operations could use luminance-weighted signals; chromatic appearance could be fitted or replaced after spatial support is established, giving monochrome-capture-plus-colorization pipelines a measured basis; and chromatic SH degree and precision could be compressed far more aggressively than luminance, which explains HiGS's 6-bit luminance / 4-bit chroma packing from the training-dynamics side.

To be clear about scale: the geometry-side gap is 0.085 dB. This is a measurement paper that clarifies a division of labor, not a performance improvement.

Limitations

The authors list nine; the load-bearing ones:

• RGB-trained geometry keeps a replicated 0.2-0.3 dB advantage on three indoor scenes, and on edges defined only by chroma, luminance-only training pulls depth toward the backdrop (post hoc; the prespecified criterion did not confirm failure).
• The capture pipeline itself weakens chroma: the benchmark JPEGs use 4:2:0 chroma subsampling (half luminance resolution per axis), and Mip-NeRF 360 images are downscaled 1.95-3.24x. The chroma-only deficit is a property of chroma as captured in these datasets; a no-subsampling control was not run.
• The causal channel study covers three scenes, several control arms are single-seed, and the depth analysis is relative to retraining variation, not ground-truth surfaces.
• Every model starts from SfM points extracted from grayscale features, so the initialization already favors luminance.
• Opacity is frozen inside the geometry set and can carry photometric information the SH solve does not reset. Conclusions are specific to standard 3DGS; sparse views and NeRFs are untested.

One more from the numbers: the formal equivalence test at ±0.15 dB narrowly fails, with the 90% interval extending 0.001 dB past the band, and the authors kept the original band. Strictly speaking, "nearly the same" is doing real work in the headline.

Terms

• 3DGS (3D Gaussian Splatting): scene representation that rasterizes millions of colored, semi-transparent 3D Gaussians directly into images.
• Spherical harmonics (SH): per-Gaussian coefficients making color vary with viewing direction; degree 0 is the base color, degrees 1-3 capture view-dependent effects.
• Densification: the training-time mechanism that clones or splits Gaussians when accumulated positional gradients cross a threshold.
• YCbCr: a color space splitting RGB into one luminance channel and two chroma channels; video codecs routinely subsample the chroma.
• Held-out PSNR: peak signal-to-noise ratio on test views not seen during training.
• LPIPS: a learned perceptual similarity metric that tracks human judgment better than PSNR.

Source

• arXiv 2610.00749v1
• PDF
• Hugging Face

What people are saying

• Study: 3DGS Geometry Is Luminance-Dominated — Chroma-Only Supervision Costs 3.9-5.5 dB kwangmoo_yi

Related papers

• Code2Video-Compass: 2,104 Tasks Test 15 MLLMs on Writing Code That Renders Whole Videos
• Column shuffle self-pairs lift variable-length scene edits to 47.6% accuracy
• Same-time reallocation lifts HunyuanVideo sparse attention by 11.8 dB PSNR
• Frozen ALPS tokens hit 85.5% on LIBERO while matching V-JEPA 2-L
• Training-free segmentation: a frozen MLLM draws B-spline masks itself, 71.5 cIoU on ReasonSeg
• Wrong tap, same login screen: LGWM hits 0.987 AUC on GUI transitions in 17 ms

All paper explainers