ARTIFICIAL INTELLIGENCE
From Gemma 3 to Gemma 4: what changed at the open-weights boundary
Gemma 4's April 2026 release changed the licensing and architecture discussion. Comparing it with Gemma 3 separates downloadable weights from reproducible training.
Research checked
Newest cited publication
A transformer diagram supplies architectural context; the article separately examines what access to model weights does and does not make reproducible.
Marxav; vectorization by Mrmw · Image source · CC0 1.0
The word open can refer to several different freedoms: running a model locally, modifying its code, redistributing its weights or rebuilding it from documented training data. Those freedoms do not necessarily arrive together. Google's transition from Gemma 3 to Gemma 4 is a timely example. Gemma 3's technical report appeared on March 25, 2025. Gemma 4 launched on April 2, 2026 under Apache 2.0, followed by a unified 12-billion-parameter variant on June 3. Comparing these generations reveals a real change in licensing and multimodal design, while preserving an important distinction: receiving capable weights does not by itself reproduce the experiment that created them.
The earlier generation provides a useful reference point
Gemma 3's report describes models with one, four, 12 and 27 billion parameters, with visual input supported by the larger variants. It combines local and global attention and uses a vision encoder to turn images into representations the language model can process. The model card distinguishes pretrained and instruction-tuned releases and documents their evaluation conditions. These details matter more than a generic claim that the family is multimodal: a specific size and checkpoint determine which inputs an application can actually send.
An image-capable language model is not simply a text model with a camera attached. A photograph must be transformed into a numerical sequence, combined with the prompt and interpreted by the learned network. Cropping, resizing and image-token allocation can change which information survives. Consider an original example: a photograph of a circuit board with tiny component labels. The model may identify the board's broad function while failing to read one printed resistance value. Recognizing the scene and extracting a precise measurement are separate tasks and should be scored separately.
The April 2026 licensing change is substantive
Google's April 2 announcement explicitly releases Gemma 4 under Apache 2.0. Gemma 3 instead sits under the separately published Gemma Terms of Use. Those terms address use and distribution of Gemma artifacts and derivatives and refer to additional use restrictions. The change should therefore be described at the artifact and generation level. A current statement about Gemma 4's permissive license cannot be retroactively substituted for the license attached to an older checkpoint merely because both have the same family name.
This is a technical publishing distinction rather than individualized legal advice. A model package may contain weights, tokenizer data, configuration files and software maintained by different parties. A useful inventory records the source and terms for each component. It also distinguishes the downloaded release from a community fine-tune that may add other obligations. My interpretation is that standard licensing can reduce friction for collaboration because reviewers recognize the permission structure. It does not answer every provenance question about the training corpus or establish that the full training process has become independently reproducible.
Parameter labels now describe different architectures
Gemma 4 launched with E2B and E4B variants, a 26B mixture-of-experts variant and a 31B dense variant. Its current model card explains that E denotes effective parameters and separates that quantity from total parameters including embeddings. It also distinguishes total and active parameters for the expert model. These are not interchangeable size measurements. An application comparing only the number in each model's name can therefore misjudge storage requirements, active computation or the memory needed to execute the model.
For a general illustration, imagine two models that each activate four billion parameters for one token. One stores four billion parameters altogether; another stores many expert blocks and chooses a subset. Their active arithmetic may be comparable while their weight-storage requirements differ greatly. Device layout and routing add further differences. A meaningful comparison reports both total and active size, precision, cache allocation and observed latency. The model card's architectural distinctions are useful precisely because they prevent a superficially simple ranking from standing in for measurements on the intended hardware.
The June model changes where visual processing happens
The June 3, 2026 release introduces Gemma 4 12B Unified. Google describes removing separate vision and audio encoders and projecting image patches and audio waveforms into the language model's embedding space through lightweight input modules. The current model card distinguishes this architecture from other Gemma 4 sizes that retain dedicated encoders. Consequently, encoder-free is a property of this variant, not a description that should be casually applied to the entire family. It also does not mean that raw media reaches the model without any preprocessing.
The architectural question is where learned feature extraction takes place. A separate encoder can transform an image into a representation before the language backbone sees it. A unified approach moves more of that burden into the shared network. Either design can succeed or fail on different inputs. My proposed comparison would use the same photographed documents, charts and short audio clips, then measure task accuracy alongside preprocessing time and peak memory. It should include noisy inputs and mismatched aspect ratios, because a clean demonstration does not reveal which pipeline component becomes the practical bottleneck.
A long context and a strong benchmark remain conditional
The published cards organize results by task, model size and training variant. Gemma 3's card, for example, includes different outcomes for visual reasoning, document questions and factuality, and identifies a limitation in its safety evaluation's language coverage. This is a reminder to read the measurement details rather than treating multimodal capability as a universal property. An application extracting information from technical drawings needs evidence on those drawings, with the actual image preparation, prompt format and answer-verification process it will deploy.
A useful original evaluation would place one crucial value in a document and ask the model to cite its location before calculating from it. Include variants where the value is absent, blurred or contradicted elsewhere. Score unsupported answers separately from incorrect arithmetic. A model that says it cannot read a label may be safer for this workflow than one that invents a plausible number. The evaluation should preserve failures and show how often human review is needed. A benchmark gain becomes actionable only when it changes the reliability or cost of the complete workflow.
Weights, permissions and scientific reconstruction are separate
The transition from Gemma 3 to Gemma 4 expands the developer's choices in meaningful ways, but it does not erase the difference between inspecting a trained system and recreating its training. A technical report can describe sources and methods without releasing every original training example, filtering decision or intermediate checkpoint. Local deployment makes execution controllable; a familiar license makes some reuse questions clearer; an open training record makes causal investigation easier. These are related benefits, and distinguishing them allows a release to receive credit for exactly what it provides.
The next persuasive independent result would pair the new architectural options with reproducible workload measurements. Publish the checkpoint identifier, software versions, media preparation and error taxonomy, then compare complete tasks under a fixed memory or time budget. Where training cannot be reproduced, state that boundary explicitly. The real lesson of the 2026 Gemma releases is that openness is something to document in concrete layers. Model users can then choose the layer of control they need, while researchers can identify which unanswered questions still depend on information beyond the released weights.
Sources & further reading
Primary papers, project documentation, and research releases used in this article. Publication dates below refer to the sources themselves.
- Gemma 3 Technical ReportGemma Team / arXivPublished Checked
- Gemma 3 model cardGoogle AI for DevelopersChecked
- Gemma Terms of UseGoogle AI for DevelopersChecked
- Gemma 4: Byte for byte, the most capable open modelsGooglePublished Checked
- Gemma 4 model cardGoogle AI for DevelopersChecked
- Introducing Gemma 4 12B: a unified, encoder-free multimodal modelGooglePublished Checked
FOLLOW THE QUESTION
Study engineering foundations