Gemma 4 introduces natively multimodal models with unified encoder-free architecture

Google introduces Gemma 4, a new generation of open-weight, natively multimodal language models featuring dense and Mixture-of-Experts architectures ranging from 2.3B to 31B parameters.

The suite includes improved vision and audio encoders for all sizes and proposes a unified, encoder-free architecture for the 12B model that ingests raw audio and image patches. It also integrates a thinking mode to generate reasoning traces prior to responding, while improving inference speed, memory, compute efficiency, and long-context abilities.

Gemma 4 establishes a leap in performance across STEM, multimodal, and long-context benchmarks, rivaling larger frontier open models in human-rated tasks.