github llama.cpp · 7h ago · inference

llama.cpp b10361 fixes SWA for EXAONE 4.5 and skips TENSOR_SKIP in metadata path

from English

The llama.cpp b10361 release addresses a critical bug where Sliding Window Attention (SWA) was not enabled for the LGAI-EXAONE 4.5 model due to incorrect parameter ordering during loading.

The model loader now correctly handles EXAONE 4.5 GGUF files by fixing the order in which `hparams.n_layer()` and `LLM_KV_NEXTN_PREDICT_LAYERS` are read, ensuring SWA is properly activated.
The metadata-only path for constructing models now skips tensors marked with `TENSOR_SKIP`, preventing assertion failures caused by null buffer types in architectures using nextn/MTP layers.
Tests were added to cover EXAONE 4 hparams ordering, verifying that the fix correctly sets `swa_type` to STANDARD rather than NONE.

This update ensures that official LGAI-EXAONE GGUF releases function correctly with sliding window attention and allows models with nextn/MTP layers to be constructed via the metadata-only path.

Importance 1/3 Trust 1/3 llama.cpp Inference efficiency

Read original

lab Hugging Face Blog · 11h ago

ALTK-Evolve achieves same-or-better accuracy than ACE at a fraction of inference cost

ALTK-Evolve, a system for agentic memory that converts agent trajectories into reusable guidelines without weight updates, matches or exceeds the accuracy of ACE while using significantly fewer tokens. Unlike ACE, which injects a comprehensive playbook on every step, ALTK-Evolve uses selective retrieval to deliver only relevant guidelines based on model capacity.

github vLLM · 14h ago

vLLM v0.27.1 adds support for quantized DSpark Markov heads

vLLM has released version 0.27.1, a patch update built on top of the previous v0.27.0 release.

lab NVIDIA Research · 15h ago · 10 views

DSTP mitigates visual token pruning failures in complex MLLM reasoning

Researchers identify Relevant Visual Information Shift (RVIS) during decoding as the primary cause for the failure of existing visual token pruning methods in Multimodal Large Language Models (MLLMs). To address this, they propose Decoding-stage Shift-aware Token Pruning (DSTP), a training-free framework that aligns visual tokens with shifting reasoning requirements.

media MarkTechPost · 16h ago · 1 view

webAI releases TwIL-LM, a 1.7B and 3B formal-logic model family for local autoformalization

webAI has released TwIL-LM, a two-model family of formal-logic reasoners with 1.7B and 3B parameters designed for autoformalization on local hardware. The models translate English into first-order logic and check conclusion validity, with the 3B variant achieving a macro gate score of 0.4218 on in-domain formal logic tasks.

media AI News (smol.ai) · 17h ago

Meta releases Muse Glimmer 30B open-weight model; Anthropic improves Riemann bound

Meta released Muse Glimmer, a 30B dense multimodal model optimized for local agent deployment under Apache 2.0, while also promising future weights for Muse Spark 1.2.