github llama.cpp · 22h ago · inference

llama.cpp b10356 updates ROCm to 7.14 and adds Windows CUDA 13 support

from English

The llama.cpp project has released version b10356, which primarily targets the build infrastructure by switching the ROCm backend from version 7.2.1 to 7.14. This update aligns with ROCm 7.14 being the first production release using TheRock build system and involves migrating CI jobs to use multi-arch wheels for both Linux and Windows.

Switches ROCm target from 7.2.1 to 7.14 across Linux and Windows builds.
Migrates shared Windows setup actions to multi-arch ROCm wheels and updates build-cuda-windows.yml accordingly.
Adds new Windows binaries for CUDA 13 (x64) and preview support for CUDA 13.4 (arm64).
Provides pre-built binaries for macOS (Apple Silicon and Intel), Linux (CPU, Vulkan, OpenVINO, SYCL), Android, and iOS.

This release ensures compatibility with the latest ROCm infrastructure while expanding GPU acceleration options on Windows through newer CUDA versions.

Importance 1/3 Trust 1/3 llama.cpp Inference efficiency

Read original

github llama.cpp · 7h ago

llama.cpp b10361 fixes SWA for EXAONE 4.5 and skips TENSOR_SKIP in metadata path

The llama.cpp b10361 release addresses a critical bug where Sliding Window Attention (SWA) was not enabled for the LGAI-EXAONE 4.5 model due to incorrect parameter ordering during loading.

lab Hugging Face Blog · 11h ago

ALTK-Evolve achieves same-or-better accuracy than ACE at a fraction of inference cost

ALTK-Evolve, a system for agentic memory that converts agent trajectories into reusable guidelines without weight updates, matches or exceeds the accuracy of ACE while using significantly fewer tokens. Unlike ACE, which injects a comprehensive playbook on every step, ALTK-Evolve uses selective retrieval to deliver only relevant guidelines based on model capacity.

github vLLM · 14h ago

vLLM v0.27.1 adds support for quantized DSpark Markov heads

vLLM has released version 0.27.1, a patch update built on top of the previous v0.27.0 release.

lab NVIDIA Research · 15h ago · 10 views

DSTP mitigates visual token pruning failures in complex MLLM reasoning

Researchers identify Relevant Visual Information Shift (RVIS) during decoding as the primary cause for the failure of existing visual token pruning methods in Multimodal Large Language Models (MLLMs). To address this, they propose Decoding-stage Shift-aware Token Pruning (DSTP), a training-free framework that aligns visual tokens with shifting reasoning requirements.

media MarkTechPost · 16h ago · 1 view

webAI releases TwIL-LM, a 1.7B and 3B formal-logic model family for local autoformalization

webAI has released TwIL-LM, a two-model family of formal-logic reasoners with 1.7B and 3B parameters designed for autoformalization on local hardware. The models translate English into first-order logic and check conclusion validity, with the 3B variant achieving a macro gate score of 0.4218 on in-domain formal logic tasks.