github llama.cpp · 2h ago · inference

llama.cpp b10618 fixes grammar parsing for literal hyphens in character classes

from English

The llama.cpp project released build b10618, which includes a fix for the grammar parser to correctly handle literal hyphens within character classes. Previously, the `gbnf_escape_char_class()` function escaped the hyphen as "\-", but the `parse_char()` function rejected this escape sequence, causing generated tool-call grammars to fail during parsing.

This update resolves the issue by allowing the parser to accept the escaped hyphen in character classes. The release also includes updated tests for this specific grammar edge case and provides binaries for macOS, Linux, Windows, Android, and openEuler across various architectures and backend implementations.

Importance 1/3 Trust 1/3 llama.cpp Inference efficiency

Read original

media Hugging Face Forums · 10h ago · 2 views

NVIDIA Nemotron-3-Omni GGUFs with working audio + video in llama.cpp (9 tested quants, one-pass A/V)

A developer has released a complete GGUF package for NVIDIA’s Nemotron-3-Nano-Omni-30B model, adding previously missing audio and video support to the llama.cpp ecosystem. The release includes nine quantizations ranging from Q3_K_M to BF16 and a unified projector file that integrates both C-RADIO vision and Parakeet audio encoders.

github llama.cpp · 11h ago · 1 view

llama.cpp b10615 adds per-device tuned flash-attn vec for Metal

The llama.cpp project released version b10615, introducing per-device tuned quantized (Q, NE) flash-attention vector operations for the Metal backend.

media Hugging Face Forums · 11h ago · 1 view

Unified GGUF and llama.cpp fork enable audio and video for Nemotron-3-Nano-Omni

The author addresses silent failures in popular GGUF versions of Nemotron-3-Nano-Omni-30B, where audio and video inputs were ignored due to incomplete projector files and missing inference graphs. To fix this, a unified mmproj file containing the vision tower, full Parakeet/FastConformer audio encoder, and temporal video embedder has been released alongside a specialized llama.cpp fork.

media Hugging Face Forums · 12h ago · 1 view

Zero-shot Nemotron 3.5 Lightning Omni adds vision and audio via geometry match

The author has created Nemotron 3.5 Lightning-Omni by attaching pre-trained projectors from NVIDIA’s Nemotron-3-Nano-Omni to the text-only Nemotron 3.5 Lightning model, enabling image and audio understanding without any additional training.

github llama.cpp · 12h ago · 1 view

llama.cpp b10614 adds Metal per-op source split and parallel compile

The llama.cpp b10614 release introduces a significant architectural change to the Metal backend by splitting operation sources and enabling parallel compilation. This update also includes several new kernel implementations and support for additional quantization formats.