Stream3D: Sequential Multi-View 3D Generation via Evidential Memory
Kaichen Zhou, Zeyang Bai, Xinhai Chang, Mengyu Wang, Paul Liang, Fangneng Zhan
cs.CV
2026-05-21
Stream3D is a training-free method that bolts a fixed-size evidential memory onto a frozen single-view 3D generator, caching the most informative past frames; on GSO it cuts reconstruction error (CD) from 0.094 to 0.048.
Today's view-conditioned 3D generators (SAM 3D, TRELLIS, Hunyuan3D and the like) reconstruct a high-quality object from a single image. But real visual observation usually arrives as a long monocular stream. The naive approach, feeding each frame to the generator independently, produces severe temporal inconsistency: the same object looks one way this frame and another way the next.
Existing latent-transport approaches try to bridge this by reusing the KV-cache or editing features in flow, but both accumulate error and grow memory linearly with sequence length, so they degrade as the stream runs long.
Stream3D is training-free and leaves the frozen generator untouched: no architectural change, no retraining, no auxiliary loss. It maintains a compact evidential memory that caches the most informative historical frames, chosen by an evidence score. The score comes from cross-attention statistics: tokens in a frame that contribute strongly to the current reconstruction with low entropy count as high evidence.
The memory is fixed-size: each token keeps only the top-D highest-evidence historical frames. As the stream advances, the memory updates dynamically while its count stays fixed, so the footprint is constant (about 65KB in the paper) and does not grow with sequence length, avoiding long-sequence degradation. Which frames feed generation is decided by aggregating token-level preferences through an ownership count and taking the top-K frames.
On the GSO and NAVI streaming benchmarks, Stream3D leads across the board. On GSO the geometric metric (Chamfer Distance, CD, lower is better) drops from SAM3D's 0.094 to 0.048, volume IoU rises from 0.664 to 0.775, and rendering quality improves too.
| Method | CD↓ | IoU↑ | Render FID↓ |
| SAM3D | 0.094 | 0.664 | 105.20 |
| SAM3D + KV-Cache | 0.084 | 0.682 | 83.35 |
| MV-SAM3D (random K frames) | 0.064 | 0.676 | 83.04 |
| Stream3D | 0.048 | 0.775 | 66.71 |
Against latent-transport baselines like KV-cache reuse and flow-based feature editing, Stream3D wins on both photometric and geometric metrics, with memory that stays constant instead of ballooning.
Single-view 3D generators are already strong but collapse on video streams. Stream3D requires no retraining of the base model, effectively adding a plug-and-play streaming shell to capable off-the-shelf generators and extending their reach from one image to a whole stream. For teams in AR, robot perception, or 3D reconstruction, this is a low-cost upgrade path.
The authors note plainly that the method's ceiling is the underlying generator. If the base model cannot reconstruct from a single view, evidential memory cannot recover missing geometry or appearance; it only spreads existing capability smoothly across time rather than creating something from nothing. Hyperparameters like the evidence score, memory depth (D), and bundle size (K) are tuned on specific benchmarks, and whether they stay optimal on streams with more aggressive motion or extreme viewpoints is not stress-tested. The training-free choice also means the weights are never specialized for streaming, so some potential may be left on the table.