Most open models assume you have a server. Google’s Gemma 3 family was built for the opposite bet: capable AI that runs on your phone, your laptop, or a $60 Raspberry Pi. The 3n variants go further, packing text, vision, and audio into a 2GB memory footprint.
In this guide I break down the real spec sheet, what actually improved over Gemma 2, verified benchmarks, and a 5-minute quickstart with Ollama and transformers so you can run it locally today.
1. The Spec Sheet: Gemma 3 and 3n at a Glance
Gemma 3 launched on March 12, 2025 in five sizes, and Gemma 3n followed with a full release on June 26, 2025 as the mobile-first branch of the family. Both are open-weights models built from the same research behind Gemini. Here is the ficha that matters for builders:
Parameters
Gemma 3 · Gemma 3n
Gemma 3: 270M, 1B, 4B, 12B, 27B. Gemma 3n: E2B (5B raw, ~2B effective) and E4B (8B raw, ~4B effective) via MatFormer nesting and Per-Layer Embedding caching.
Context Window
32K · 128K
128K tokens for Gemma 3 4B/12B/27B, 32K for 1B/270M and both 3n variants. Enough for long documents and multi-image conversations on-device.
Modalities
Text · Image · Audio · Video
Gemma 3 handles text + images. Gemma 3n adds audio (speech recognition and translation) and video understanding through a MobileNet-V5 vision encoder.
Languages
140+
Text coverage in over 140 languages, with multimodal understanding across 35. Strong gains in Japanese, German, Korean, Spanish, and French.
License
Gemma Terms of Use
Open weights with responsible commercial use permitted. Note: the newer Gemma 4 family (April 2026) switched to Apache 2.0 — 3/3n stay on Gemma terms.
Price
Free weights · $0 inference
Weights are free on Hugging Face and Kaggle. Self-hosted inference costs nothing per token — you pay only for your own hardware or electricity.
One Honest Note on Versions
Google released Gemma 4 in April 2026 (E2B, E4B, 26B MoE, 31B, Apache 2.0) plus a 12B unified multimodal model in June 2026. It is the newest family — but Gemma 3/3n remains the edge-proven pick with the widest tooling: Ollama, llama.cpp, MLX, LiteRT, and AI Edge Gallery all support it today.
2. What Actually Improved Over Gemma 2
Gemma 3 was not just a size bump. The jump from Gemma 2 to Gemma 3/3n changed what you can ship on constrained hardware. These are the improvements I verified in the official docs and model cards:
Effective Parameters (3n)
Per-Layer Embeddings plus MatFormer nesting let a 5B model run with a ~2GB footprint (E2B) and an 8B model with ~3GB (E4B). You get big-model quality at small-model RAM.
Native Audio Input
Gemma 3n is the first Gemma that hears: on-device transcription and speech translation with no internet. Gemma 2 and base Gemma 3 are text/vision only.
Faster Mobile Vision
The MobileNet-V5 encoder runs up to 13x faster quantized on a Pixel TPU than Gemma 3’s previous vision stack, with 46% fewer parameters and 60 fps video throughput.
Mix-n-Match Elasticity
One E4B checkpoint can be sliced into custom sizes between E2B and E4B to fit your exact latency budget — no separate downloads per size.
Arena-Level Quality
Gemma 3n E4B passed 1300 LMArena Elo — the first model under 10B parameters to do it — while Gemma 3 27B competes with models far above its weight.
Verified Benchmarks (official model cards)
Scores below come from Google’s published evaluations at full precision. Instruction-tuned where marked IT, pre-trained where marked PT:
- • HellaSwag 10-shot: 78.6 (E4B PT) · 72.2 (E2B PT)
- • ARC-c 25-shot: 61.6 (E4B PT) · 51.7 (E2B PT)
- • HumanEval 0-shot: 75.0 (E4B IT) · 66.5 (E2B IT)
- • MMLU 0-shot: 64.9 (E4B IT) · 60.1 (E2B IT)
- • WMT24++ translation (ChrF): 50.1 (E4B IT)
- • MBPP 3-shot: 63.6 (E4B IT) · 56.6 (E2B IT)
3. Quickstart: Run It in 5 Minutes
You have two verified paths. Ollama is fastest for trying it now; transformers gives you full multimodal control (image + audio) in Python. Both use the official checkpoints — nothing custom.
Step 1 — Ollama (fastest, text + vision)
Install Ollama from ollama.com/download, then pull the official build. The E2B tag fits almost any laptop:
ollama pull gemma3n # effective 2B — smallest footprint ollama run gemma3n:e2b # effective 4B — best quality ollama run gemma3n:e4b
Step 2 — transformers (full multimodal)
You need transformers 4.53.0 or newer. This snippet runs the official E4B checkpoint with an image input, straight from the Hugging Face model card:
pip install -U transformers
from transformers import pipeline
pipe = pipeline(
"image-text-to-text",
model="google/gemma-3n-e4b-it",
device=0,
)
out = pipe(
"https://huggingface.co/datasets/huggingface/"
"documentation-images/resolve/main/bee.jpg",
text="<image_soft_token> in this image, there is",
)
print(out[0]["generated_text"])Step 3 — Serve It as a Local API
Ollama exposes an OpenAI-style local endpoint, so your app talks to localhost instead of a cloud API. Verified against the official Ollama integration guide:
curl http://localhost:11434/api/generate -d '{
"model": "gemma3n",
"prompt": "summarize this release note in 3 bullets"
}'Tip: Pick the Right Size First Try
Phone or Raspberry Pi 5 8GB: start with gemma3n:e2b (~2GB effective). Laptop with 16GB: go straight to E4B. Desktop with a GPU: Gemma 3 12B is the quality jump. You can Mix-n-Match between E2B and E4B later without re-downloading.
4. Honest Limits
I run small models on real hardware, so here is what the announcement posts underplay. None of these are dealbreakers — but you should know them before you architect around Gemma 3n:
✅ Where It Shines
- • Offline-first apps: chat, summarization, and RAG with zero per-token cost
- • Mobile multimodal: captioning, visual Q&A, and transcription on-device
- • Privacy-sensitive apps: health, kids, enterprise data that cannot leave the device
- • Prototyping multimodal features without an API bill
- • Fine-tuning a compact open model for one domain task
❌ Where It Struggles
- • Heavy reasoning and competition math: AIME 2025 sits at 11.6 (E4B) — use a cloud frontier model instead
- • Long-context beyond 32K on 3n: Gemma 3 4B+ gives you 128K, 3n does not
- • Ollama on Pi exposes text only: full multimodal needs the transformers pipeline, slower on ARM
- • Knowledge cutoff June 2024: anything newer needs retrieval or a newer checkpoint
- • Multimodal on low RAM: loading vision/audio params raises the footprint above the 2–3GB headline
Golden rule
Match the model to the constraint, not the hype. If your app must work on a plane, in a hospital, or on a $60 board, Gemma 3n is the best open option I have tested. If you need frontier reasoning, pay for the cloud — no 4B model will save you.
5. Verdict: Should You Build on It?
Yes — if your constraint is the device, not the datacenter. Gemma 3n E2B is my default recommendation for anything that must run offline on a phone or a Raspberry Pi 5: it loads fast, leaves RAM headroom, and its ~6 tokens/sec on a Pi 5 is workable for background tasks, assistants, and smart-home parsing.
For laptops, step up to E4B or Gemma 3 4B QAT. For maximum quality on a server, Gemma 3 27B — or evaluate the newer Gemma 4 family under Apache 2.0. The Gemmaverse momentum (400M+ downloads, 100K+ variants) means fine-tunes and tooling keep compounding.
My Verdict
Gemma 3n is the first open model that makes on-device multimodal feel practical instead of experimental. Run the Ollama quickstart above tonight — you will know within 10 minutes whether it fits your app.
Conclusion
The Gemma 3 family — and especially 3n — moved the frontier from the cloud to your pocket. Realistic sizes, real multimodal inputs, and a license that permits commercial use make it the most shippable open option for edge AI in 2026.
Start with the 5-minute Ollama run, then go deeper with transformers when you need vision and audio. And if you outgrow 3n, the path to Gemma 4 is already paved with the same tooling.
Cheat Sheet: Summary
Run It
- • ollama run gemma3n:e2b (phone / Pi)
- • ollama run gemma3n:e4b (laptop)
- • transformers ≥ 4.53.0 (multimodal)
Remember
- • E2B ≈ 2GB · E4B ≈ 3GB effective
- • 32K context · 140+ languages
- • Cutoff June 2024 — add RAG
Next Steps
- • AI Edge Gallery (Android demo)
- • LiteRT / Google AI Edge (ship it)
- • Evaluate Gemma 4 (Apache 2.0)
