SLAMSqueezeBench: Comparing SLAM Systems under Resource Constraints
Mohamed Hefny, Karthik Dantu, Steven Y. Ko
cs.RO
2026-09-17
SLAMSqueezeBench tests nine SLAM systems under live frame rates and resource caps. Gaussian methods drop 94%+ frames even uncapped; ATE hides stalls that SPE scores.
On a robot, SLAM never owns the machine. Cameras emit at 25–30 Hz while planning, segmentation, and control share the same CPU and GPU. Standard benchmarks replay a log by waiting for each frame to finish, so a slow system still sees every image and posts a clean trajectory error.
SLAMSqueezeBench removes that courtesy. It caps CPU, RAM, disk, GPU memory, and GPU compute with cgroups and HAMi-core, then adds neighbors: stress-ng, a custom SqueezeGPU matmul burner, or SAM 3. Frames arrive at the dataset’s nominal rate; a two-frame buffer drops the oldest when full. Alongside ATE, the paper scores Stale Pose Error (SPE): at every ground-truth timestamp it uses the latest pose the system had produced, so gaps grow with distance travelled.
Nine systems run on the sequence each family supports. Classical geometric methods (ORB-SLAM3 with and without IMU, OKVIS2-X, cuVSLAM) use EuRoC V101 at 20 Hz, 500 frames. Learning methods (DROID-SLAM, DPV-SLAM, MASt3R-SLAM, VGGT-SLAM) use TUM fr1/desk at 30 Hz. Gaussian systems (GigaSLAM, S3PO-GS) use KITTI 07 at 10 Hz, 200 frames. Hardware is fixed: i7-12700K, 62 GiB, RTX 3090. Each setting is repeated three times.
E1 tightens a CPU cap down to half a core. E2 matches that average CPU use with competing workers, no cap. E3/E4 add synthetic GPU load and SAM 3. E5 applies desktop (8 cores / 16 GiB / 8 GiB GPU) and embedded (6 / 4 / 4) budgets together. E6 squeezes only ORB-SLAM3: 30% clean, 20% half-core, 50% released.
The deadline alone already splits the field. Uncapped with live delivery, ORB-SLAM3 without IMU drops 15% of EuRoC frames; learning systems drop 16–75% on TUM; GigaSLAM drops 94% on KITTI and S3PO-GS drops 97% then fails. cuVSLAM reports 0% loss, ATE about 0.070 m, and 499–500 poses from no deadline through a half-core cap.
ATE lies. Under a half-core cap VGGT-SLAM still posts 0.021 m ATE, the same as at four cores, while pose count falls from 41 to 17 and SPE rises from 0.313 m to 0.858 m. The gaps vanish from ATE and remain in SPE.
Matching average CPU is not matching a cap. ORB-SLAM3 with IMU fails at a one-core cap but finishes beside 30 CPU workers that leave it about one core on average. DPV-SLAM goes the other way: the same average CPU with workers yields 283 poses and 0.594 m ATE versus 327 and 0.497 m under the cap.
GPU neighbors scale with duty cycle. DROID-SLAM’s pose count falls from 61 to 26 as SqueezeGPU duty cycle goes from 0.25 to 0.95 on 4096 matrices, with frame loss 56% to 93%. VGGT-SLAM reports 34 poses at every duty cycle. SAM 3 holds the GPU about 77% busy; the six GPU systems lose about as many frames as they did at duty 0.75–0.95. On the embedded budget DROID-SLAM, MASt3R-SLAM, VGGT-SLAM, GigaSLAM, and S3PO-GS all fail. In the transient squeeze, ORB-SLAM3 lost tracking and reset the map in five of six runs; after release it tracks again, but the saved trajectory starts after the cap lifts.
Picking an onboard SLAM system from a centimeter ATE table is the wrong test. This paper turns “did frames arrive, did poses keep coming, did the map survive the squeeze” into a shared protocol. SPE is the number a controller actually lives with. Both Gaussian systems miss the camera at 10 Hz on KITTI, which is closer to a vehicle install than another indoor PSNR.
The three families run on three datasets and three rates, so cross-family accuracy is not comparable. Caps are software on a desktop, not a Jetson memory hierarchy. E6 covers only ORB-SLAM3. SPE uses the starting ground-truth pose before the first estimate, folding startup delay into the score. Neither SqueezeGPU nor SAM 3 is a full navigation stack. The code is promised at publication; no third-party rerun yet.