Honeycomb: Constant-Size Scene Memory Representation for Video World Models
Jack Wei Lun Shi, Kaichen Zhou, Haoyu Chen, Yufeng Weng, Keane Ong, Ruojin Cai, Hang Hua, Justin K. W. Yeoh, Mengyu Wang
cs.CV
2026-09-29
Honeycomb stores video world model scene memory in six fixed planes, warp-and-fuse instead of growing; revisit flow error 3.00 px vs Spatia's 6.64, storage constant.
Video world models generate footage along camera trajectories, and short clips look fine. The trouble starts on long rollouts: when the camera circles back to where it started, the layout and furnishings should match the opening frames, but those early observations have long since fallen out of the generator's temporal context. Current fixes store observations in explicit spatial memory. Spatia keeps an updatable RGB point cloud; LSM-World caches diffusion latents attached to 3D points. Both work, and both keep growing as generation proceeds. Honeycomb (Harvard, NUS, MIT, and MIT-IBM Watson AI Lab; code public) reframes the question: can scene memory stay a fixed size and still recognize the scene on a revisit?
The core is HexMemory, a scene representation factorized into six fixed-size 2D feature planes: three spatial (XY, XZ, YZ) and three spatiotemporal (XT, YT, ZT). The decomposition comes from HexPlane, but HexPlane fits its planes per scene by gradient descent; here a shared feed-forward writer builds and updates them instead. Generation proceeds autoregressively in overlapping chunks of 33 frames at 704×1280.
The pipeline loops three steps:
Three design choices carry the weight. Each point carries its latent token, camera center, ray direction, and write time, which a small network maps to per-plane contributions splatted bilinearly into four neighboring cells. Accumulated interpolation weights act as confidence maps that decide how much old and new planes contribute at fusion. Training optimizes a single latent reconstruction loss jointly across writer, fusion networks, and reader, which pushes the planes to retain exactly what a revisit needs. Unlike Spatia and LSM-World, dynamic objects and sky are not filtered out of memory writes.
On WorldScore (3,000 image-to-video samples, evaluated without dynamic object filtering), Honeycomb averages 68.01 against 63.21 for Spatia, 61.20 for LSM-World, and 55.21 for Wan2.1, the best general video model in the table.
| Setting | Method | Metric | Result |
| RE10K novel-view synthesis | Honeycomb / Spatia | PSNR | 18.45 dB / 15.58 dB (LSM-World 17.46) |
| RE10K novel-view synthesis | Honeycomb / Spatia | LPIPS | 0.274 / 0.390 |
| WorldScore closed-loop revisit | Honeycomb / next-best Voyager | PSNR | 17.22 dB, +1.23 dB |
| WorldScore closed-loop revisit | Honeycomb / Spatia | flow error | 3.00 px / 6.64 px |
| Memory write | recurrent / per-scene optimization | per chunk | 13 ms constant / 3.2 s, 240× slower |
The ablations sharpen the picture. Dropping plane resolution from 512 to 256 shrinks storage from 73.9 MB to 19.8 MB (73% less) at a cost of 0.12 dB closed-loop PSNR; at 128 (5.8 MB) the drop is 0.45 dB. A replacement writer that rebuilds planes from all stored points each chunk matches quality (17.16 vs 17.22) but its write time grows with the rollout, reaching 40 ms by chunk 9, while the recurrent writer stays at 13 ms. Per-scene direct optimization scores slightly higher (17.44) at roughly 240× the write cost.
For anyone building long-horizon generation or interactive world models, this memory has two properties the alternatives lack: storage and write cost are both independent of scene scale. Trading 0.12 dB for 19.8 MB shows the capacity-quality curve is flat, so the low-rank planes compress scene information gracefully. The six planes are directly visualizable; corridor and staircase structure shows up in the feature maps, which beats staring at a point cache during debugging. Call it architectural consolidation rather than a conceptual leap: HexPlane's factorization existed, and the contribution is reshaping it into a feed-forward, recurrently updatable form trained jointly with a diffusion generator.
The paper admits it: as bounds expand, the fixed grids cover a larger range at coarser resolution. Constant size has a built-in capacity ceiling, and the ablations only run to chunk 9, so much longer rollouts are untested. On WorldScore, Subject Const sits at 46.28 versus 77.18 for Wan2.1 and 46.66 for Spatia, making dynamic subjects the weakest axis, plausibly tied to writing unfiltered dynamics into memory. Depth and poses come entirely from monocular estimation (ViPE with Depth Anything 3), so estimation errors land directly in memory, and the paper does not analyze how they accumulate over a rollout. Training and evaluation stay on RealEstate10K real-estate footage, so generalization to other domains is unknown. Direct optimization still scores 0.22 dB higher; the 240× speedup is bought with a little quality.