Jag: direct box prediction makes compact multimodal models faster at visual grounding
Xiuyuan Zhu · hf · 2026-10-15
Researchers from UCAS and Peng Cheng Lab present Jag, which adapts a compact pretrained multimodal model for direct continuous box prediction instead of autoregressive coordinate decoding.
- A lightweight regression head reads the last valid input-token state, conditioned on the image and referring expression, predicting all four box coordinates in one forward pass
- Geometric supervision jointly adapts the language backbone, visual merger, and regression head
- Across benchmarks, Jag consistently improves grounding accuracy over its base model; ablations show joint adaptation beats training the head alone
- It matches specialized grounding models' accuracy with lower single-request latency and GPU memory
More from Multimodal
- Alibaba TaoLive open-sources 3-step LoRA for real-time streaming audio-video generation — linoy_tsaban · 2026-10-15
- Entire video generated by Opus 5.5 on Max reasoning, no skills used — Hesamation · 2026-10-15
- Argon writes its own 3D raytracer, video tool and audio mixer — FinanceYF5 · 2026-10-15
- Microsoft's AesCode-32B learns aesthetics from image generators to code rich posters and dashboards — Powerful_Evening5495 · 2026-10-15
- One prompt, 15 minutes: Fable 5.5 delivers a cool animated short — FinanceYF5 · 2026-10-15
- AI Filmmakers Release Episode 2 of Cinematic Short 'The Third Force' — No-Past-7449 · 2026-10-15