Google’s DiffusionGemma shows a retrofit path for text diffusion models

Google’s DiffusionGemma text diffusion model, introduced June 10 as an experimental open release, converts an existing Gemma 4 model to parallel text generation rather than pretraining a new system from the ground up. The result matters for teams pursuing low-latency local inference, though Google says the 26-billion-parameter mixture-of-experts model produces lower-quality output than standard Gemma 4 and is not its recommended choice where quality is the priority.

Released under Apache 2.0, DiffusionGemma is built from the Gemma-4-26B-A4B base model, according to Google and a subsequent technical report covered by The Decoder. The report says the conversion used less than 10% of the original model’s training-token budget. That is a useful demonstration that an autoregressive model can be adapted into a text-diffusion system without repeating full pretraining. It does not establish that retrofit training will be the best approach for every architecture, dataset or production workload.

How does DiffusionGemma generate text differently?

Conventional autoregressive language models emit tokens in sequence, with each new token conditioned on those already generated. DiffusionGemma instead drafts and repeatedly refines blocks of 256 tokens in parallel. This gives each token access to the rest of the block during refinement, a property Google says is useful for inline editing, code infilling and other tasks where later text constrains earlier text.

The reported training process has two stages. First, the model learns to reconstruct noisy text blocks. It then undergoes a combined reinforcement-learning and sampler-distillation stage, called SD·RL, intended to improve response quality while reducing the number of refinement steps required. The Decoder reports that DiffusionGemma also retains an autoregressive generation mode, giving developers a choice of decoding approach.

Where does the speed advantage apply?

Google claims up to four times faster generation on dedicated GPUs: more than 1,000 tokens per second on a single Nvidia H100 and more than 700 on an RTX 5090. Those are company measurements, not independent validation. The Register’s review of Google’s comparisons reported a roughly 2.25-times speedup against Gemma 4 12B with speculative decoding and nearly four times against Gemma 4 26B-A4B on one H100.

The condition is deployment shape. Google positions the model for single-user or low-to-medium-batch local inference, where sequential decoding can leave hardware capacity unused. In high-QPS cloud serving, autoregressive systems can batch many requests efficiently. Google says diffusion’s advantage then diminishes and may raise serving costs; The Decoder reports that conventional models catch up in throughput at about 32 concurrent requests.

What should operators take from the release?

DiffusionGemma is an engineering option rather than a replacement for conventional LLMs. Google describes it as experimental and says standard Gemma 4 remains the better fit when output quality is paramount. Fine-tuning results on Sudoku suggest that bidirectional block generation can help with tightly constrained tasks, but they do not show broad reasoning superiority.

For teams considering it, the relevant test is workload-specific: local interactive latency, acceptable output quality and the cost of expected concurrency. That is the distinction a proper AI model evaluation should measure, rather than treating a headline speed figure as a general deployment result.

This story draws on original reporting from The Decoder.