media r/LocalLLaMA · 79d ago · open_models

Gemma 4 E2B runs at 255 tok/s in browser using WebGPU

from English

Gemma 4 E2B achieves 255 tokens per second in-browser on an M4 Max using WebGPU kernels. The demo and kernels are now available on Hugging Face for public use.

Importance 2/3 r/LocalLLaMA Google DeepMind API & product launches Inference efficiency Open weights

Read original

media r/LocalLLaMA · 51d ago · 4 views

Google updates Gemma 4 chat templates with tool calling fixes and Flash Attention 4 support

Google has released an update to the Gemma 4 model that modifies its chat templates. The changes address significant issues with tool calling capabilities and reduce "laziness" in the model's responses.

media r/LocalLLaMA · 52d ago

Specific Labs steers Gemma-4-31B to push back on false premises without benchmark impact

Specific Labs has released Gemma-4-31B-AntiHal, a variant of the Gemma-4-31B model that challenges requests based on false premises rather than hallucinating along with them. The model achieves this through interpretability-based representation steering applied to the residual stream during generation.

media r/LocalLLaMA · 54d ago · 2 views

Zer0Fit wraps Google's TabFM and TimesFM as a local MCP server

A developer has created Zer0Fit, an open-source implementation that serves Google's TabFM and TimesFM foundation models as a Model Context Protocol (MCP) server for zero-shot machine learning tasks. The project allows users to perform forecasts, classifications, and regressions locally without building or training custom models.

media r/LocalLLaMA · 71d ago · 1 view

Gemma4-26B-A4B & 31B-QAT Uncensored Balanced Released with MTP Speed Boosts

HauhauCS has released two new uncensored, balanced versions of the Gemma 4 models: Gemma4-26B-A4B and Gemma4-31B-QAT. Both variants incorporate Multi-Token Prediction (MTP) draft heads to enable speculative decoding, resulting in significant inference speed improvements. The 26B-A4B model achieves approximately a 35% speed boost, while the 31B model sees a 53% increase, with identical output quality verified by the model's drafting mechanism. These releases utilize QAT-aware quantization, making Q4_K_M the optimal format as higher precision offers no quality gains for these specific models. The 26B-A4B is a Mixture of Experts architecture with roughly 4 billion active parameters per token, whereas the 31B variant is a dense model offering higher capability for users with sufficient VRAM. Both models include vision support via mmproj files and maintain a 262K context window. The author notes that GenRM testing resulted in zero refusals across 465 prompts, confirming their uncensored nature.

github llama.cpp · 76d ago · 6 views

llama.cpp Release b9741 Adds New Binaries and Support

llama.cpp version b9741 introduces new binaries for macOS, Linux, Android, Windows, and openEuler across multiple architectures. The release includes support for Vulkan, CUDA 12.4 and 13.3, OpenVINO, SYCL, and ROCm, with updated versions for iOS and Ubuntu.