media Hugging Face Forums · 4d ago · open_models

Undergrad proposes architectural changes to Qwen 2.5 Omni 3B to stop visual interference

from English

An undergraduate student on the Hugging Face forums discusses applying concepts from the VAPO paper to the Qwen 2.5 Omni 3B model to address hallucination issues during audio transcription with slide context.

The VAPO paper identifies that flattening and concatenating different token types in Omni-modal Large Language Models causes visual interference, leading to hallucinated text in transcripts.
VAPO solves this using a reinforcement learning paradigm with a 'temporally decoupled policy' that separates visual prior extraction from transcription.
The student asks if architectural modifications to Qwen 2.5 Omni 3B, such as capturing embeddings before the thinker block or altering the thinker block itself, can prevent visual tokens from interfering with audio tokens.

The post seeks technical guidance on implementing these decoupling strategies within the existing Qwen architecture.

Importance 1/3 Trust 1/3 Hugging Face Forums Multimodal Research paper

Read original

arxiv arXiv cs.CL · 27d ago · 45 views

VisCAD releases foundation model suite with multimodal industrial CAD intelligence

Researchers present VisCAD, a foundation model suite designed to provide broad generalization and strong capability for realistic industrial product design. The suite addresses the challenges of part-level generation from diverse inputs like renders and text, as well as assembly-level generation involving interacting parts.

arxiv arXiv cs.AI · 49d ago · 30 views

SCOUT improves spatial reasoning in VLMs via structured CoT and process-supervised RL

Researchers propose SCOUT, a framework that enhances Vision-Language Models' spatial reasoning by combining structured Chain-of-Thought with multi-objective process-reward reinforcement learning. This approach addresses credit assignment issues in existing RL methods and integrates depth perception for comprehensive 3D understanding.

lab Microsoft Research Blog · 50d ago · 40 views

Microsoft Research introduces CARE-X, a unified chest X-ray VLM with auxiliary supervision and tool-augmented measurement

Microsoft Research has introduced CARE-X, a research model designed to address gaps in current radiology vision-language models by combining generative report writing with calibrated diagnostic predictions. The system uses reinforcement learning (DAPO) to reward clinical correctness and pairs the Qwen3-VL-4B-Instruct model with deterministic measurement tools to evaluate performance on conditions requiring precise calculations.

lab NVIDIA Research · 50d ago · 31 views

GenRecal distills large vision-language models to small ones via recalibration

Researchers present GenRecal, a general-purpose distillation framework that transfers knowledge from large vision-language models (VLMs) to smaller, more efficient counterparts. The method addresses the challenge of heterogeneous VLM architectures by using a Recalibrator to align and adapt feature representations across different model types.

lab NVIDIA Research · 50d ago · 34 views

Hide to See introduces reasoning-prefix masking for VLM distillation

Researchers propose a novel think-answer distillation framework that improves compact vision-language models' ability to utilize visual evidence by masking salient reasoning prefixes. This approach addresses the "visual forgetting" issue common in long think-answer traces, where students lose focus on visual cues during extended reasoning.