Pumpire: benchmarking physical point-pair distance in 3D foundation models
Siyu Chen · hf · 2026-10-09
Pumpire is a unified benchmark evaluating metric point-pair distance estimation of image- and video-level 3D foundation models, with or without depth priors.
- Prior work evaluates depth and camera intrinsics separately, or scores point clouds with geometric similarity metrics — neither directly measures point-to-point distance ability. Pumpire assesses distances directly from reconstructed geometry.
- Ships the pumpire-6k dataset: 100 real-world scenes, 64 frames each (6,400 total), annotated with physically measured point-pair distances.
- Extensive experiments across 29 baseline configurations of representative 3D foundation models, enabling direct error assessment and cross-setting comparison.
The goal is to test physical-scale perception in reconstructed 3D space, a fundamental ability prior protocols overlooked. Project page: pumpire.github.io
More from Research
- EA-VAE paper in IEEE T-PAMI fixes systematic uncertainty failures in VAEs — enzoferrante · 2026-10-10
- OpenAI reportedly solved 92 of the 500 most important open math problems in one GitHub push — altryne · 2026-10-10
- AI math is teleportation to a foggy summit: Cepelewicz's striking mountain metaphor — burny_tech · 2026-10-10
- Isola lays out the three main pushbacks to the PRH narrative his new paper answers — phillip_isola · 2026-10-10
- Three findings from Isola's new paper: global map, text-to-image translation, orthogonality — phillip_isola · 2026-10-10
- Isola's team shows a global orthogonal map aligns image and text embeddings without paired data — phillip_isola · 2026-10-10