Gemma 4

model
Google

Four-variant family released under Apache 2.0 (a notable license upgrade from prior Gemma releases). Includes a 31B dense flagship, 26B-A4B MoE (4B active), and edge variants E4B and E2B. 256K token context for the larger models. Multimodal: text, image, video, audio.

Architecture features alternating attention, dual RoPE, Per-Layer Embeddings (PLE), and shared KV cache. Gemma 4 31B achieves 85.2% MMLU Pro, 89.2% AIME 2026, 80.0% LiveCodeBench v6, and Codeforces Elo 2,150. Ranked #3 open model on Arena AI text leaderboard. Crossed 2M downloads shortly after release.

AnnouncementHuggingFaceApple ML Research

Model Details

Architecture DENSE
Parameters 31B
Context window 256,000

Variants

Name Parameters Notes
Gemma 4 31B 31B Dense flagship, 256K context
Gemma 4 26B-A4B 26B MoE, 256K context
Gemma 4 E4B 8B Effective 4.5B, 128K context
Gemma 4 E2B 5.1B Effective 2.3B, 128K context
open-weightopen-sourcemultimodalmoe

Related