DBRX Instruct

Databricks · DBRX132B70B+Databricks Open Model License chat code

Databricks' 132B mixture-of-experts — 16 experts, 4 active per token (36B active params). Trained on 12T tokens on Mosaic infrastructure and released under the Databricks Open Model Licence. DBRX was best-in-class on release; now beaten by Llama 3.3 70B and Qwen 2.5 72B on most benchmarks, but retains value as a well-documented MoE reference.

Parameters: 132B
Context length: 33K
Modality: text
Released: 2024-03-27

Memory & hardware

VRAM (fp16): 264 GB
VRAM (Q4): 79.2 GB
Recommended: 4× A100 80GB or 2× H100 80GB
Quantizations: fp16, q8_0, q4_k_m

License: Databricks Open Model License

SPDX: —
Commercial use: Yes
Modification: Yes
Redistribution: Yes

License detail →

Benchmarks

MMLU

73.7

HumanEval

70.1

MMLU-Pro

55.8

MATH

34.6

Benchmarks last verified 2026-07-02.

Hosted inference pricing

No hosted pricing listed — this model is currently self-host-only on this site.

Run it yourself

Drop-in commands for the three most common open-source inference paths. The Ollama tag is a best-effort match against the registry; verify the size variant before pulling.

Run DBRX Instruct locally

No official Ollama registry tag for this model — use transformers or vLLM below.

vLLM (production)

vllm serve databricks/dbrx-instruct

High-throughput hosted inference; one command to expose an OpenAI-compatible HTTP server.

Transformers (Python)

from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("databricks/dbrx-instruct")
model = AutoModelForCausalLM.from_pretrained(
    "databricks/dbrx-instruct", device_map="auto", torch_dtype="auto"
)

Direct PyTorch usage. Pin a torch / cuda version that matches your GPU.

Hugging Face ID: databricks/dbrx-instruct

Related models

Same family or similar size — useful when shopping around.

Mixtral 8×22B Instruct

141B

Scaled-up Mixtral with 22B-parameter experts. ~39B active parameters out of 141B total. Strong long-context performance and competitive coding scores. Apache 2.0 makes it attractive for self-hosting where the licence terms of Llama 3 are a non-starter.

Context: 66K
License: apache-2-0
VRAM Q4: 84.6 GB

Command R+

104B

Cohere's flagship 104B model. RAG-focused with native multilingual support across ~10 high-resource languages. CC-BY-NC weights; commercial use via Cohere's hosted API.

Context: 128K
License: mrl
VRAM Q4: 62.4 GB

Llama 3.2 90B Vision

90B

Larger vision-language Llama variant, competitive with the proprietary multimodal frontier on standard image-understanding benchmarks. Drops in as a vision upgrade where 11B isn't sharp enough. Requires substantial GPU memory in fp16; most teams will run it quantized or on multi-GPU. A natural pairing with retrieval pipelines that fetch image-rich chunks alongside text.

Context: 128K
License: llama-3
VRAM Q4: 54 GB

Qwen2.5 72B Instruct

72B

The flagship Qwen 2.5 release. Competes with Llama 3.1 405B on many benchmarks at one-fifth the parameter count. Note the 72B specifically uses the Qwen License (commercial use up to 100M MAU) — the smaller Qwen2.5 sizes are Apache 2.0.

Context: 128K
License: qwen
VRAM Q4: 43.2 GB

DeepSeek R1 Distill Llama 70B

70B

R1 reasoning capabilities distilled into a Llama 3.3 70B base. The most accessible way to run R1-class reasoning locally — fits on a single H100 in fp16 or on a 4090 at Q4. Inherits Llama 3's community licence (commercial use under 700M MAU). Great pick for production reasoning workloads where the full R1 is too expensive to host but o1/R1-style quality is required.

Context: 128K
License: llama-3
VRAM Q4: 42 GB

Llama 3.1 Nemotron 70B Instruct

70B

NVIDIA's RLHF-tuned Llama 3.1 70B. Tops several Arena-style human-preference leaderboards and shipped with NVIDIA's reward-model research. Inherits the Llama 3 community licence.

Context: 128K
License: llama-3
VRAM Q4: 42 GB