media r/LocalLLaMA · 1h ago · open_models

G9v3-39A5B MoE model shows strong results on artificialanalysis

from English

The G9v3-39A5B model is drawing attention for its performance on artificialanalysis, where it reportedly ranks above Qwen 3.6 27B.

It is a Mixture of Experts (MoE) architecture.
A llama.cpp feature branch exists to support the G9v3 format.
GGUF checkpoints are available via Hugging Face.

Importance 2/3 r/LocalLLaMA Inference efficiency Local inference

Read original

media Hugging Face Forums · 1h ago · 1 view Live

Celestium pipeline enables SDXL on 4GB-8GB GPUs via adaptive FP16 and CPU fallback

A diagnostic using Celestium’s hardware-agnostic inference pipeline demonstrates that Stable Diffusion XL can run stably on low-VRAM GPUs (4GB, 6GB, and 8GB) by leveraging adaptive FP16 precision and autonomous memory cleanup. The approach prevents fragmentation spikes during multi-stage pipelines and utilizes an emergency CPU mode to maintain generation when GPU VRAM is exhausted.

github llama.cpp · 11h ago · 1 view

llama.cpp b10505 adds dedup-cache-models preset option

The llama.cpp project has released version b10505, which introduces a new `dedup-cache-models` preset option for the server component.

media Hugging Face Forums · 2d ago · 3 views

Fourier Magnitude KV Cache Quantization doubles context length at same memory

ntrillard demonstrates that quantizing the Fourier magnitude spectrum of K and V cache values to 4-bit while preserving phase at full precision achieves 96.9% token match with fp16 reference on Gemma-3-1B, a 5-10x improvement over standard min-max quantization.

github llama.cpp · 5d ago · 26 views

llama.cpp adds Kimi-K3 model support with MXFP4 repacking and chat template

The llama.cpp project has added support for the Kimi-K3 text model, implementing its hybrid KDA (linear) + MLA (full) attention architecture along with five specific architectural features not present in the older Kimi-Linear-48B.

github llama.cpp · 5d ago · 21 views

llama.cpp b10447 redesigns yield_to_queue thread model

The llama.cpp project has released build b10447, which includes a re-design of the `yield_to_queue` thread model. This change involves running `common_speculative_process` in the worker and swapping the worker-to-main thread design.