Gemma 2 2B
Compact Gemma variant designed for on-device inference. Trained with knowledge distillation from larger Gemma 2 teachers. Runs comfortably on a phone at Q4.
- Parameters
- 2.6B
- Context length
- 8K
- Modality
- text
- Released
- 2024-07-31
Memory & hardware
- VRAM (fp16)
- 5.2 GB
- VRAM (Q4)
- 1.6 GB
- Recommended
- CPU or any GPU
- Quantizations
- fp16, q8_0, q4_k_m
Benchmarks
Hosted inference pricing
No hosted pricing listed — this model is currently self-host-only on this site.
Run it yourself
Drop-in commands for the three most common open-source inference paths. The Ollama tag is a best-effort match against the registry; verify the size variant before pulling.
ollama run gemma2:3b
vllm serve google/gemma-2-2b-it
from transformers import AutoTokenizer, AutoModelForCausalLM
tokenizer = AutoTokenizer.from_pretrained("google/gemma-2-2b-it")
model = AutoModelForCausalLM.from_pretrained(
"google/gemma-2-2b-it", device_map="auto", torch_dtype="auto"
)google/gemma-2-2b-it Related models
Same family or similar size — useful when shopping around.
The smallest Llama 3 release, designed for on-device inference on phones and laptops. The 1B model runs comfortably in <2 GB of RAM at Q4 quantization and is fast enough for real-time chat on a modern smartphone. Useful for edge inference, on-device assistants where round-tripping to a server is undesirable, and as a draft model for speculative decoding in front of a larger Llama 3 variant.
- Context
- 128K
- License
- llama-3
- VRAM Q4
- 0.6 GB
Mid-tier Gemma. Strong general-purpose chat model at small scale. The Gemma Terms of Use permit commercial use subject to Google's prohibited-use policy.
- Context
- 8K
- License
- gemma
- VRAM Q4
- 5.4 GB
Flagship Gemma 2 release. Uses logit-distillation from a larger teacher model, which is how Google delivers near-70B quality from a 27B student. A solid choice when the Llama community licence doesn't fit and you need quality at the 27B–40B size range.
- Context
- 8K
- License
- gemma
- VRAM Q4
- 16.2 GB