LLMSeg: turning off-the-shelf MLLMs into text-prompted segmenters, no training
ducha_aiki · x · 2026-10-22
KAUST researchers introduce LLMSeg, a training-free pipeline that turns a vision-capable LLM into a text-prompted image segmenter with no segmentation head, mask decoder, or training.
- Method: three LLM-driven stages — detect localizes instances matching a free-form query; segment traces each instance with closed periodic B-splines confined to its box (outer boundary plus optional holes); refine iteratively proposes focus boxes around residual errors and retraces.
- The model only outputs coordinates (boxes and B-spline control points) as JSON; parsing, rasterization, and stopping are deterministic code, with overlaid masks fed back each pass.
- Results: across 21 MLLMs tested, better vision+reasoning means better segmentation; any decent MLLM can segment. The B-spline loops actually converge to more accurate masks.
- Paper, code and website are public.
More from Multimodal
- Synthesia CEO calls Syren "the ChatGPT moment for video" — synthesiaIO · 2026-10-22
- Hermes Agent demo: rigged, dancing game characters from one prompt each, no Blender — alexcovo_eth · 2026-10-22
- AI researcher proposes x0 velocity, a new training target for flow-matching diffusion models — ostrisai · 2026-10-22
- Seedance 2.5 AI-Generated Video Dubbed an 'Absolute Masterpiece' — nikola_mr64990 · 2026-10-22
- Prompt template: 'Abstract Symbolist Reverberation' for moody oil-paint AI art — LudovicCreator · 2026-10-22
- Kokon 0.10 reuses Qwen-Image 2.1's text encoder to write prompts, runs on 8GB cards — Ok-Director4174 · 2026-10-22