Mistral Nemo 12B
Joint Mistral × NVIDIA model with 128K context, designed as a drop-in upgrade to Mistral 7B. Trained with NVIDIA's Megatron stack and released under Apache 2.0. Strong multilingual coverage thanks to the Tekken tokenizer.
- Parameters
- 12B
- Context length
- 128K
- Modality
- text
- Released
- 2024-07-18
Memory & hardware
- VRAM (fp16)
- 24 GB
- VRAM (Q4)
- 7.2 GB
- Recommended
- RTX 4090 24GB
- Quantizations
- fp16, q8_0, q5_k_m, q4_k_m
Benchmarks
Hosted inference pricing
USD per million tokens.
| Provider | Input | Output | |
|---|---|---|---|
| deepinfraCheapest | $0.13 | $0.13 |
Run it yourself
Drop-in commands for the three most common open-source inference paths. The Ollama tag is a best-effort match against the registry; verify the size variant before pulling.
ollama run mistral-nemo
vllm serve mistralai/Mistral-Nemo-Instruct-2407
from transformers import AutoTokenizer, AutoModelForCausalLM
tokenizer = AutoTokenizer.from_pretrained("mistralai/Mistral-Nemo-Instruct-2407")
model = AutoModelForCausalLM.from_pretrained(
"mistralai/Mistral-Nemo-Instruct-2407", device_map="auto", torch_dtype="auto"
)mistralai/Mistral-Nemo-Instruct-2407 Related models
Same family or similar size — useful when shopping around.
The original Mistral 7B refresh with 32K context and extended vocabulary. Permissive Apache 2.0 weights and the first widely-deployed sliding-window-attention model. Still useful in 2026 for very-low-cost inference and as a baseline for fine-tuning experiments.
- Context
- 33K
- License
- apache-2-0
- VRAM Q4
- 4.2 GB
Stability AI's general-purpose 12B model. Apache 2.0. Useful default when you need a permissively-licensed 12B-scale model.
- Context
- 4K
- License
- apache-2-0
- VRAM Q4
- 7.2 GB
Llama 3's first vision-language model. Image understanding via a separately-trained ViT adapter bolted onto Llama 3 weights. Useful for OCR-adjacent workloads, document understanding, and image captioning at a permissive licence. The 11B size makes it cheap to host. Combined with the 128K text context, it handles long PDF-with-images workflows comfortably on a single 4090.
- Context
- 128K
- License
- llama-3
- VRAM Q4
- 6.6 GB
Mid-tier Gemma. Strong general-purpose chat model at small scale. The Gemma Terms of Use permit commercial use subject to Google's prohibited-use policy.
- Context
- 8K
- License
- gemma
- VRAM Q4
- 5.4 GB
The workhorse 8B instruction-tuned model. Excellent quality-to-cost ratio and the broadest ecosystem support of any open-weights model — every major inference engine, fine-tuning library, and quantization toolchain has a 3.1 8B preset. Fits in 24 GB of VRAM at fp16, ~6 GB at Q4. Strong default for production chat where 70B is overkill, for fine-tuning on a specialist task, and for any workload where you want a known-good baseline.
- Context
- 128K
- License
- llama-3
- VRAM Q4
- 4.8 GB
Apache-2.0-licensed 7B model with surprisingly strong reasoning and multilingual chops. Qwen 2.5 trains on a larger and more carefully filtered corpus than the original Qwen series, and the 7B variant punches well above its weight on coding and math benchmarks. A strong default for cost-sensitive chat workloads and for fine-tuning experiments where the Apache licence simplifies downstream redistribution.
- Context
- 128K
- License
- apache-2-0
- VRAM Q4
- 4.2 GB