Microsoft has released Mage-VL, an efficient 4B-parameter multimodal foundation model for image and video understanding that uses a codec-native approach to streamline visual processing. The system separates video streams into anchor (I) frames and predicted (P) frames, retaining only patches where the codec allocates bits to reduce visual token consumption by over 75%.
- Mage-VL pairs a from-scratch Codec-ViT visual encoder with a Qwen3-4B causal decoder language backbone.
- The architecture achieves up to 3.5× wall-clock inference speedup over uniform frame sampling while preserving spatio-temporal context.
- A System 1 & System 2 dual-process design adds proactive streaming, using a lightweight cognition gate to invoke the full VLM only when necessary.
- On video benchmarks, Mage-VL improves over Qwen3-VL-4B on every reported metric, with significant gains in localization-heavy tasks like +22.5 on QVHighlight.
The model targets the trade-off between complex offline reasoning and real-time streaming perception, offering a single-model solution for low-latency event-gated commentary.