SAM-MT: Real-Time Multi-Object Video Segmentation
FudanCVL · hf · 2026-07-11
This work proposes SAM-MT for real-time interactive multi-object video segmentation. The authors point out that while existing methods perform well on single targets, extending them to multiple targets usually requires repeating the entire single-target pipeline for each object. This tanks the frame rate, with latency becoming uncontrollable as targets increase.
Method
- Modifies Segment Anything 2 (SAM2) into a multi-object interaction framework
- Uses explicit queries to represent different targets while retaining global context in parallel
- Reduces interference between targets and maintains identity distinction via decoupled masked attention
- Combines sparse memory for temporal stability, adding occlusion handling and overlap prevention strategies
Results
- Decouples latency from the number of targets
- Maintains >36 FPS even with 10 targets
- Performance approaches single-target baselines while preserving SAM2's video segmentation quality
More from Multimodal
- Gemini Omni Flash turns a boat cabin into a cave in Flow by Google — chrisfirst · 2026-07-22
- A simple workflow to turn a photo into an image prompt using Gemini, Grok, or GPT Image — harshitagu72595 · 2026-07-22
- A Reddit user proposes a consistency LoRA to keep anime and game scenes visually stable — ThirdWorldBoy21 · 2026-07-22
- Hand-painted figurines run through Seedance look eerily alive — cocktailpeanut · 2026-07-22
- An AI agent-made bayou country music video is making the rounds on Reddit — LazyKaleidoscope4696 · 2026-07-22
- Testing Qwen 3 Image: Map Borders Shift Based on Prompts, Includes Chinese Labels — NirantK · 2026-07-22