github llama.cpp · 1h ago · inference

llama.cpp b10218 adds minicpmv46 downsample support

from English

The llama.cpp project released build b10218, which includes the addition of minicpmv46 downsample support via pull request #25993. This update introduces a 4x ignore VIT merger and places the downsample mode inside the GGUF format.

Added minicpmv46 downsample functionality to mtmd_image_preprocessor_llava_uhd.
Integrated the downsample mode directly into the GGUF structure.
Included a 4x ignore VIT merger in the conversion process.

This release provides binaries for macOS, Linux, Windows, Android, and openEuler across various hardware accelerators including CUDA, ROCm, Vulkan, and OpenVINO.

Importance 1/3 Trust 1/3 llama.cpp Inference efficiency Multimodal

Read original

media r/LocalLLaMA · 4d ago · 1 view

Microsoft releases Mage-VL, a codec-native streaming multimodal model

Microsoft has released Mage-VL, an efficient 4B-parameter multimodal foundation model for image and video understanding that uses a codec-native approach to streamline visual processing. The system separates video streams into anchor (I) frames and predicted (P) frames, retaining only patches where the codec allocates bits to reduce visual token consumption by over 75%.

github llama.cpp · 6d ago

llama.cpp b10142 adds preliminary MiniMax-M3 vision support

The llama.cpp project released build b10142, which introduces preliminary vision support for the MiniMax-M3 model. This update implements a text-only port that reuses existing components from MiniMax-M2, including GQA with per-head QK-norm and partial rotary, DeepSeek-V3 style experts, and swigluoai activation.

media r/LocalLLaMA · 17d ago · 2 views

OvisOCR2 tops OmniDocBench as first end-to-end document parsing model

ATH-MaaS released OvisOCR2, a 0.8B vision-language model post-trained from Qwen3.5-0.8B that serves as the first end-to-end model to top the OmniDocBench leaderboard with a score of 96.58 on v1.6.

arxiv arXiv cs.AI · 19d ago

Output token count, not visual processing, dominates energy in edge VLM inference

A systematic energy profiling of on-device Vision-Language Model (VLM) inference across five models and two hardware platforms reveals that the primary energy bottleneck is output generation rather than visual perception. The study overturns the common assumption that reducing visual tokens is the most effective efficiency strategy, demonstrating instead that controlling output length is critical for power savings.

arxiv arXiv cs.CL · 25d ago

OpenGlass introduces sensing-computing split architecture for local MLLM-driven visual assistance

Researchers present OpenGlass, an open-source system that splits sensing and computing to provide low-latency multimodal visual assistance for blind and low-vision users. The architecture uses an ESP32-based glasses unit to capture visual context while a nearby consumer device performs local MLLM inference, keeping raw data on user-controlled devices by default.