github llama.cpp · just now · inference

llama.cpp b10514 adds conversion support for GraniteSWA and GraniteMoeSWA models

from English

The llama.cpp project has released version b10514, which introduces conversion infrastructure for IBM's GraniteSWAForCausalLM and GraniteMoeSWAForCausalLM architectures. This update enables users to convert these specific model types into the GGUF format for use with llama.cpp.

Added feat(convert) support for GraniteSWAForCausalLM and GraniteMoeSWAForCausalLM.
Implemented conversion infrastructure for rope_pattern arrays and per-layer rope determination.
Fixed SWA pattern logic to support non-rope layers and corrected naming for ffn gate inp.
Updated llama_hparams with has_rope constants and removed hacky rope_finetuned fallbacks.
Included binaries for macOS, Linux, Windows, and Android across CPU, CUDA, Vulkan, OpenCL, and other backends.

This release allows the llama.cpp ecosystem to natively support IBM's sliding window attention models, expanding the range of compatible architectures available for local inference.

Importance 1/3 Trust 1/3 llama.cpp IBM Inference efficiency

Read original

media Hugging Face Forums · 1h ago · 1 view Live

Measurements suggest revisiting q_proj/v_proj default for LoRA target modules

The author measures gradient subspace overlap across four models to evaluate the standard LoRA default of targeting query and value projections. The analysis indicates that this default may be suboptimal when the goal is retaining previous capabilities rather than maximizing accuracy on a new task.

github llama.cpp · 5h ago · 1 view

llama.cpp b10509 adds ggml_rope_set_offset support to Metal, CUDA, and Vulkan

The llama.cpp project released build b10509, which introduces support for the `ggml_rope_set_offset` function across multiple hardware backends. This update includes CPU, Metal, CUDA, and Vulkan implementations of the new operation.

github llama.cpp · 9h ago · 1 view

llama.cpp b10505 adds dedup-cache-models preset option

The llama.cpp project has released version b10505, which introduces a new `dedup-cache-models` preset option for the server component.

lab Liquid AI · 21h ago · 1 view

LFM2.5 Q4_0 checkpoints use Quantization-Aware Distillation to recover 97% of BF16 accuracy

The LFM2.5 team has released updated 4-bit GGUF checkpoints for the LFM2.5-230M, LFM2.5-350M, LFM2.5-1.2B-Instruct, and LFM2.5-2.6B models, trained using Quantization-Aware Distillation (QAD). This technique distills a high-precision teacher model into a quantized student to recover accuracy lost during standard quantization.

lab Hugging Face Blog · 22h ago · 2 views

Liquid AI releases LFM2.5 Q4_0 checkpoints via Quantization-Aware Distillation

Liquid AI has released Q4_0 GGUF checkpoints for its LFM2.5 model family, utilizing a new training method called Quantization-Aware Distillation (QAD). This approach distills high-precision teacher models into quantized student models to recover accuracy lost during standard post-training quantization.