arxiv arXiv cs.AI · 1h ago · research

ClinFusion introduces vision-centric MLLM for holistic medical understanding

from English

Researchers introduce ClinFusion, a vision-centric multimodal large language model designed to address the challenges of deploying AI in clinical practice by unifying 2D and 3D medical image understanding. The system features a compositional cascaded vision encoder with a Cascade Spatial-Aware Locality Fusion operator and includes a new evaluation framework comprising MedIF-Bench and region-of-interest-grounded metrics.

ClinFusion sets a new state-of-the-art across 20 out of 24 benchmarks, outperforming open-source models like Hulu-Med and Lingshu.
It demonstrates multimodal capabilities superior to proprietary models GPT-5.2 and Gemini-3-Flash on 13 out of 16 benchmarks.
The model supports agentic tool use for retrieval-augmented and tool-assisted clinical workflows.
Blinded evaluations by board-certified radiologists confirm ClinFusion produces the highest-ranked reports.
The proposed RoI-grounded metric shows the strongest correlation with expert judgment among automatic evaluation metrics.

The authors consider this significant because it provides a clinically aligned, factualness-driven assessment method and validates that their model's output aligns closely with expert radiologist judgment.

Importance 2/3 arXiv cs.AI Multimodal Research paper

Related articles

arxiv arXiv cs.CL · 4h ago

ClinFusion: vision-centric MLLM sets new state of the art in medical benchmarks

Researchers introduce ClinFusion, a vision-centric multimodal large language model designed for holistic medical understanding that unifies 2D and native 3D image analysis. The system utilizes a compositional cascaded vision encoder with a Cascade Spatial-Aware Locality Fusion operator to address the challenges of heterogeneous medical imaging.

lab NVIDIA Research · 11h ago

Multimodal AI analysis of non-verbal communication in Delta Force

Researchers analyze gameplay videos from the competitive shooter game Delta Force to extract and understand emergent non-verbal communication, such as gestures and movement patterns. This work addresses a gap in prior studies that have largely focused on MOBA games or verbal coordination in other shooters.

arxiv arXiv cs.CL · 12d ago

CSSEL-P2P enables simultaneous speech translation without architectural changes

The article presents CSSEL-P2P, a data-driven approach for simultaneous speech translation that avoids the brittle architectural changes and explicit read/write policies often required by decoder-only LLM systems. The method utilizes fixed-length chunks for cumulative streaming decoding with a rewind-based committed prefix, combined with teacher-labeled prefix-to-prefix targets and bounded waiting for fine-tuning.

arxiv arXiv cs.CL · 12d ago

Groc-PO introduces grounded context preference optimization to reduce multimodal hallucinations

Researchers propose Grounded Context Preference Optimization (Groc-PO), a framework designed to address untruthfulness issues like visual hallucinations and content fabrication in Multimodal Large Language Models (MLLMs). The method tackles the credit-assignment challenge where standard Direct Preference Optimization (DPO) fails to suppress error propagation from early grounding stages by applying supervision only at the final answer.

arxiv arXiv cs.CL · 13d ago

Interactive multi-feature fusion improves semantic reconstruction from non-invasive brain recordings

A new framework addresses the representational mismatch in continuous semantic reconstruction by fusing static lexical representations (W2V) with dynamic contextualized representations (GPT). The study benchmarks two integration approaches: linear Naive Concatenation and non-linear Multi-Head Cross-Attention.