Tsinghua's KBMR embeds images by semantic entity, boosting knowledge-based visual QA

Tsinghua · hf · 2026-09-03

Tsinghua researchers introduced KBMR, a retrieval method for knowledge-based visual question answering.

• Instead of surface visual similarity, KBMR uses a multimodal language model to embed images by semantic identity, aligning embeddings with the underlying entity rather than appearance.
• Training relies on continuous distillation and hard negative sampling to sharpen retrieval.
• The authors report improvements in both retrieval quality and downstream VQA accuracy.

Original post →

More from Multimodal

• Ex-Brain Games host made a 25-min AI film in 5 days using 'vibe directing' in iMessage — lmoroney · 2026-10-21
• Motion Launches a General Video Agent That Edits Footage and Mimics Any Reference Style — mhdfaran · 2026-10-21
• What keeps AI videos watchable: hooks, consistency and a payoff — aftahi_ai · 2026-10-21
• MORPHJUTSU skill in Flova transfers any video's motion to your character — aftahi_ai · 2026-10-21
• NVIDIA open-sources MotionBricks: 350K motion skills at 15,000 FPS, 2ms latency — aftahi_ai · 2026-10-21
• Runway joins Claude Marketplace, letting enterprises spend Anthropic budget on video generation — runwayml · 2026-10-21

Multimodal channel →