SAM-MT: Real-Time Multi-Object Video Segmentation
FudanCVL · hf · 2026-07-11
This work proposes SAM-MT for real-time interactive multi-object video segmentation. The authors point out that while existing methods perform well on single targets, extending them to multiple targets usually requires repeating the entire single-target pipeline for each object. This tanks the frame rate, with latency becoming uncontrollable as targets increase.
Method
- Modifies Segment Anything 2 (SAM2) into a multi-object interaction framework
- Uses explicit queries to represent different targets while retaining global context in parallel
- Reduces interference between targets and maintains identity distinction via decoupled masked attention
- Combines sparse memory for temporal stability, adding occlusion handling and overlap prevention strategies
Results
- Decouples latency from the number of targets
- Maintains >36 FPS even with 10 targets
- Performance approaches single-target baselines while preserving SAM2's video segmentation quality
More from Multimodal
- TischEins, a large Midjourney SREF library, relaunches with new design after security outage — tisch_eins · 2026-10-26
- MOSS-Image-SFT 4-step distillation tops 16.5K downloads on Hugging Face — _akhaliq · 2026-10-26
- Claude Code can now edit videos: a 10-prompt workflow from raw footage to final cut — TawohAwa · 2026-10-26
- One-sentence 3D worlds turn cinematic with CineMap and TapNow — gizakdag · 2026-10-26
- No Camera Needed: Creator Builds an Entire World in CineMap and Directs It Into Cinema — gizakdag · 2026-10-26
- Kling vs. Seedance: the Chinese AI video model rivalry — DavidmComfort · 2026-10-26