ClinFusion uses vision-first multimodal LLMs to raise medical benchmark results

Alibaba-DAMO-Academy · hf · 2026-07-28

ClinFusion is a vision-centric multimodal LLM system for holistic medical understanding.
The paper focuses on a medical setting where the main challenge is absorbing heterogeneous 2D and 3D medical images and evaluating outputs in a way that matches radiologists’ clinical practice.
It proposes a compositional, cascaded vision encoder with a Cascade Spatial-Aware Locality Fusion operator, plus a vision-grounded evaluation framework including MedIF-Bench and a region-of-interest grounded report-generation metric.
The system reports new SOTA across a broad set of medical VQA, report generation, instruction-following, and textual medical tasks, and blinded radiologist review ranks its reports highest.

More from Multimodal