Generative Semantic Scene Completion
Shi Chen, Weifeng Ge
cs.CV, cs.LG, cs.RO
2026-08-27
S2D2 refines a frozen SSC model in one discrete-diffusion step, lifting SCPNet from 36.7% to 38.8% mIoU on the SemanticKITTI hidden test without retraining.
A single LiDAR sweep hits about 1% of the target voxel grid. Outdoor semantic scene completion (SSC) has to fill the rest and label 19 classes at once. SemanticKITTI uses a 256×256×32 grid over 51.2×51.2×6.4 m at 0.2 m. Empty voxels are about 95% of the volume. Among labelled points, vegetation is 26.7% and motorcyclist is 0.0037%, more than a 7,000× gap.
Discriminative nets train on that histogram. Rare classes, thin poles, ghost trails, boundaries, and occlusions all fail in the same places. SCPNet's hidden-test per-class IoU tracks training frequency at Pearson r=0.80: the road users a planner needs most are the ones the set supplies least. Reweighting the loss or squeezing more labels from the same sequences does not move the histogram.
GSSC runs one multinomial discrete diffusion in three jobs. Discrete diffusion resamples voxel classes through a transition matrix, not by adding Gaussian noise.
On the hidden test, frozen SCPNet plus one S2D2 step with no test-time augmentation reaches 38.8% mIoU, +2.1 pp over SCPNet's published 36.7%. The paper indexes that as the strongest causal, single-sweep, single-sample leaderboard result. Four steps with eight-view TTA hit 39.2%, outside that predicate. Four-sweep SCPNet at 47.5% and TALoS at 37.9% (test-time adaptation) sit outside it too.
On val, their SCPNet port scores 36.17%; one step reaches 38.54% (+2.36). SemCity refines the same base's released weights with a continuous prior, 37.55% to 38.19%, and gains more on completion IoU (+9.0 vs +2.8 here). Treat them as comparable, not ranked. From-noise SGSC scores 30.5% on val, 32.8% after a S2D2 pass, still 3.3 pp short of the discriminative base.
| Method | Split | mIoU | Completion IoU |
| SCPNet published | test | 36.7 | 56.1 |
| TALoS | test | 37.9 | 60.2 |
| SCPNet+S2D2 (N=1) | test | 38.8 | 58.9 |
| SCPNet+S2D2 (N=4+TTA) | test | 39.2 | 59.0 |
| SCPNet port | val | 36.17 | 49.9 |
| +S2D2 (N=1) | val | 38.54 | 52.7 |
| SGSC from noise | val | 30.5 | 53.6 |
| SGSC+S2D2 | val | 32.8 | 55.4 |
The operator lifts three frozen bases: LMSCNet +1.8, JS3C-Net +1.6, SCPNet +2.36. On SCPNet, 18 of 19 classes rise; bicyclist and truck each +5.3, road +4.6; pole, fence, and trunk barely move. Cross-entropy without the KL term collapses to 10.3%. KL alone sits at 36.1%, the base's own score, so it does nothing. N=1/4/100 land at 38.54 / 38.65 / 38.2. One step is enough.
This is a correction operator for a deployed SSC stack, not another completion network to retrain. SCPNet's official code still depends on spconv 1.0, which is gone from PyPI, and this paper did not resubmit the ported weights to the hidden test. An existing prediction can serve as the source; one forward pass buys about two mIoU points. The correction costs 107 ms; the full pipeline runs at 3.23 FPS on an idle H100 against the base's 4.95. Code and the PS3 pool are public.
From-noise generation still loses to a discriminative base. The usable piece is the third role: correcting voxel labels a stack already produces.
The refiner cannot exceed its source. On JS3C-Net, VRU-IoU falls 3.8 to 2.4 (person 8.7 to 5.9, bicyclist 2.6 to 1.2). One val frame drops person IoU from 36.6 to 0.7%. Weaker bases also lose building and truck. Transfer is scoped to voxel-grid-native sources.
Motorcyclist's +8.3 on val does not reproduce: +0.3 on retrain, 1.8 to 3.1 on the hidden test. On the safety mean the paper introduces, S3CNet leads at VRU-IoU 32.6 against 21.6. Thin structures barely move. Occlusion is never isolated.
Latency is not repaid. At a 1 s reaction window, DW-VRU-IoU falls 50.7 to 49.9 on val and 54.9 to 49.7 on test: the refined pipeline trails the frozen base. On test, the accuracy gain does not cover the extra time. Zero-shot SSCBench-KITTI360 mIoU moves 5.8 to 6.2, inside single-seed noise; only completion IoU (18.1 to 19.5) is a real claim. SemanticPOSS moves mIoU 1.0 to 6.5 and completion IoU 31.8 to 54.9, on reconstructed rather than official ground truth, so it is not comparable to TALoS.
Their SCPNet port sits 1.0 pp under the paper's 37.2% val figure and 1.4 pp under the released weights SemCity cites. The hidden-test +2.1 is versus SCPNet's published score, not versus a weight file they can reproduce.