github llama.cpp · just now · inference

llama.cpp b10369 adds pocket-tts support with 80% faster CUDA generation

from English

The llama.cpp project released build b10369, introducing support for the pocket-tts text-to-speech model. This update includes a new implementation of transposed convolutions using GEMM and col2im to optimize performance.

Generation time per frame drops by 80% on CUDA and by 50% on CPU compared to previous implementations.
The output matches the reference implementation with a correlation of 0.999994 and identical frame counts.
Language packs now tune end-of-speech padding and short prompt padding via new mmproj keys.
A long French text test showed generation within 2% of the reference time (22.96s vs 23.44s).

The optimization significantly accelerates audio synthesis while maintaining fidelity, and existing mmproj files must be converted to carry the new configuration keys.

Importance 1/3 Trust 1/3 llama.cpp Inference efficiency Voice & audio

Read original

media r/LocalLLaMA · 28d ago · 3 views

audio.cpp 0.3 adds Supertonic 3, MOSS-TTS, IndexTTS2, and Irodori-TTS with GGUF support

The audio.cpp project has released version 0.3, introducing five new text-to-speech models: Supertonic 3, MOSS-TTS-Local, MOSS-TTS-Nano, IndexTTS2, and Irodori-TTS. The update also adds GGUF support, which will be rolled out model by model.

media r/LocalLLaMA · 33d ago

Qwen3-ASR and Kokoro-TTS ONNX on CPU enable GPU-free voice assistants

A user tested the latency of Qwen3-ASR and Kokoro-TTS running as ONNX models on CPU to determine how much processing load can be offloaded from the GPU in a voice assistant pipeline.

media r/LocalLLaMA · 34d ago

audio.cpp adds streaming support and Nemotron 3.5 ASR

The audio.cpp project has updated its native C++/GGML framework to include initial streaming support for speech recognition and synthesis, alongside four new ASR/STT models: Nemotron 3.5 ASR, Higgs Audio STT, VibeVoice ASR, and Hviske ASR.

media r/LocalLLaMA · 35d ago · 1 view

Developer builds Android app for on-device Qwen3 TTS inference

A developer has created an Android application that runs the Qwen3TTS 0.6B model locally on mobile devices. The app utilizes a custom GGML-based C++ backend to enable text-to-speech processing without cloud dependency.

media r/LocalLLaMA · 38d ago

audio.cpp adds native C++ VibeVoice 1.5B support for local audio inference

The author of the audio.cpp runtime has added support for the VibeVoice 1.5B model, enabling long-form multi-speaker text-to-speech generation in a native C++/ggml environment.