Llama 3.1 405B Instruct
Meta's July 2024 flagship — the first open-weights model at 405B parameters. Trained on 15T tokens with 128K context. Rivals GPT-4o on many academic benchmarks and set the ceiling for open-weights quality for most of 2024. Running it self-hosted requires serious hardware (8× H100 at fp8 or multi-node at fp16); most users will run it via a hosted provider (Together, Groq, Fireworks). Llama 3.3 70B closed most of the practical gap at a fraction of the cost, so 405B is now most useful when 70B specifically hits its ceiling.
- Parameters
- 405B
- Context length
- 128K
- Modality
- text
- Released
- 2024-07-23
Memory & hardware
- VRAM (fp16)
- 810 GB
- VRAM (Q4)
- 243 GB
- Recommended
- 8× H100 80GB at fp8, or hosted via Together/Groq/Fireworks
- Quantizations
- fp16, fp8, q8_0, q4_k_m
License: Llama 3 Community License
- SPDX
- —
- Commercial use
- Yes
- Modification
- Yes
- Redistribution
- Yes
Benchmarks
Hosted inference pricing
USD per million tokens.
| Provider | Input | Output | |
|---|---|---|---|
| together | $3.50 | $3.50 | |
| groq | $2.15 | $2.15 | |
| deepinfraCheapest | $0.80 | $0.80 |
Run it yourself
Drop-in commands for the three most common open-source inference paths. The Ollama tag is a best-effort match against the registry; verify the size variant before pulling.
ollama run llama3.1:405b
vllm serve meta-llama/Llama-3.1-405B-Instruct
from transformers import AutoTokenizer, AutoModelForCausalLM
tokenizer = AutoTokenizer.from_pretrained("meta-llama/Llama-3.1-405B-Instruct")
model = AutoModelForCausalLM.from_pretrained(
"meta-llama/Llama-3.1-405B-Instruct", device_map="auto", torch_dtype="auto"
)meta-llama/Llama-3.1-405B-Instruct Related models
Same family or similar size — useful when shopping around.
Larger vision-language Llama variant, competitive with the proprietary multimodal frontier on standard image-understanding benchmarks. Drops in as a vision upgrade where 11B isn't sharp enough. Requires substantial GPU memory in fp16; most teams will run it quantized or on multi-GPU. A natural pairing with retrieval pipelines that fetch image-rich chunks alongside text.
- Context
- 128K
- License
- llama-3
- VRAM Q4
- 54 GB
Meta's December 2024 refresh of Llama 3 70B that closes most of the gap with Llama 3.1 405B for chat workloads while remaining tractable on a single H100. Strong instruction following, robust tool-use behaviour, and a 128K context window make it the default choice for production chat at 70B scale. The 3.3 release was trained on a refreshed instruction-tuning data mix and benefits from Meta's most recent alignment work. It outperforms the much larger 3.1 405B on several reasoning benchmarks at a fraction of inference cost. The licence is the Llama 3 Community License, which permits commercial use unless your service exceeds 700M monthly active users. Good pick for: production chat at scale, RAG over long documents, agentic workflows where tool use matters, and any 70B-tier replacement for closed proprietary models.
- Context
- 128K
- License
- llama-3
- VRAM Q4
- 42 GB
The pre-3.3 70B workhorse. Same base architecture as Llama 3.3 70B but the earlier instruction-tuning recipe. Still widely referenced as a baseline in papers and provider docs, and still the default 70B on some hosted providers.
- Context
- 128K
- License
- llama-3
- VRAM Q4
- 42 GB
Flagship Llama 2 release. Fundamentally superseded by Llama 3 70B on every benchmark, but relevant historically: the model that made 'open-weights chat model at frontier scale' credible for enterprise workloads.
- Context
- 4K
- License
- llama-2
- VRAM Q4
- 42 GB
Hybrid Mamba-Transformer-MoE model with native 256K context (effective beyond 140K). 94B active parameters out of 398B total. The state-space-model layers give it linear-time scaling with sequence length, making it interesting for very long contexts. Licensed under AI21's open model licence, which permits most commercial use.
- Context
- 256K
- License
- jamba-open
- VRAM Q4
- 238.8 GB
NVIDIA's reward-modelling research vehicle. Trained primarily to be a synthetic-data-generation specialist rather than a chat-first model. Useful for teams building instruction-tuning datasets at scale.
- Context
- 4K
- License
- llama-3
- VRAM Q4
- 204 GB