Feature
Open-source AI models with long context windows (100K+)
Long-context models can hold entire books, codebases or transcripts in one prompt without RAG. Watch the caveat: effective context (where the model reliably attends to all of it) is often shorter than the advertised window.
30 models with long context
Moonshot AI's 1-trillion-parameter mixture-of-experts (32B active per token). Trained on 15.5T tokens with a heavy emphasis on tool-use and agentic behaviour. Modified-MIT licence with an attribution clause for very-large deployments. Exceptional at long-horizon agent tasks; benchmarked well against Claude Sonnet on SWE-bench Verified.
- Context
- 128K
- License
- kimi
- VRAM Q4
- 600 GB
671B-parameter MoE model with 37B active per token. Trained for roughly $5.6M of compute — a landmark in cost-efficient frontier training. Frontier-class quality at a fraction of the cost of the closed proprietary frontier. The DeepSeek licence permits commercial use with limited restrictions on military and unlawful applications. Running V3 yourself requires serious hardware (8× H100 at fp8); most teams will use it via the DeepSeek API or providers like Together.
- Context
- 128K
- License
- deepseek
- VRAM Q4
- 402.6 GB
Reasoning model trained with reinforcement learning on top of DeepSeek V3-Base. MIT licence — even the weights are unrestricted, making R1 the most permissively-licensed frontier reasoning model. Generates long internal chains-of-thought before answering, trading latency for accuracy on math, code, and reasoning benchmarks. Distilled variants (e.g. R1 Distill Llama 70B) recover most of the quality at much smaller scales.
- Context
- 128K
- License
- mit
- VRAM Q4
- 402.6 GB
Meta's July 2024 flagship — the first open-weights model at 405B parameters. Trained on 15T tokens with 128K context. Rivals GPT-4o on many academic benchmarks and set the ceiling for open-weights quality for most of 2024. Running it self-hosted requires serious hardware (8× H100 at fp8 or multi-node at fp16); most users will run it via a hosted provider (Together, Groq, Fireworks). Llama 3.3 70B closed most of the practical gap at a fraction of the cost, so 405B is now most useful when 70B specifically hits its ceiling.
- Context
- 128K
- License
- llama-3
- VRAM Q4
- 243 GB
Hybrid Mamba-Transformer-MoE model with native 256K context (effective beyond 140K). 94B active parameters out of 398B total. The state-space-model layers give it linear-time scaling with sequence length, making it interesting for very long contexts. Licensed under AI21's open model licence, which permits most commercial use.
- Context
- 256K
- License
- jamba-open
- VRAM Q4
- 238.8 GB
xAI's second open-weights release, Apache 2.0. ~300B mixture-of-experts. xAI's pattern of open-sourcing the previous frontier when a new one ships continues from Grok 1. Competitive with GPT-4-class chat quality at release; today useful mainly as a research artefact given the compute needed to run it.
- Context
- 131K
- License
- apache-2-0
- VRAM Q4
- 180 GB
Coding-focused MoE model with 21B active parameters out of 236B total. Supports 338 programming languages with strong performance across mainstream stacks (Python, TypeScript, Go, Rust, Java, C++) and competent results on niche languages where most open models falter. The DeepSeek licence applies — commercial use permitted with some application restrictions.
- Context
- 128K
- License
- deepseek
- VRAM Q4
- 141.6 GB
The flagship Qwen 3 release: a 235B-total MoE with 22B active parameters per token. Competitive with DeepSeek V3 and Llama 4 Maverick on reasoning benchmarks while being smaller total. Apache 2.0 — one of the most permissively licenced frontier-class models.
- Context
- 128K
- License
- apache-2-0
- VRAM Q4
- 141 GB
Cohere's flagship 104B model. RAG-focused with native multilingual support across ~10 high-resource languages. CC-BY-NC weights; commercial use via Cohere's hosted API.
- Context
- 128K
- License
- mrl
- VRAM Q4
- 62.4 GB
Larger vision-language Llama variant, competitive with the proprietary multimodal frontier on standard image-understanding benchmarks. Drops in as a vision upgrade where 11B isn't sharp enough. Requires substantial GPU memory in fp16; most teams will run it quantized or on multi-GPU. A natural pairing with retrieval pipelines that fetch image-rich chunks alongside text.
- Context
- 128K
- License
- llama-3
- VRAM Q4
- 54 GB
The flagship Qwen 2.5 release. Competes with Llama 3.1 405B on many benchmarks at one-fifth the parameter count. Note the 72B specifically uses the Qwen License (commercial use up to 100M MAU) — the smaller Qwen2.5 sizes are Apache 2.0.
- Context
- 128K
- License
- qwen
- VRAM Q4
- 43.2 GB
The pre-3.3 70B workhorse. Same base architecture as Llama 3.3 70B but the earlier instruction-tuning recipe. Still widely referenced as a baseline in papers and provider docs, and still the default 70B on some hosted providers.
- Context
- 128K
- License
- llama-3
- VRAM Q4
- 42 GB
Larger Hermes 3 variant on top of Llama 3.1 70B. Widely used in agent-heavy workloads that need strong tool use combined with reliable function-calling schemas.
- Context
- 128K
- License
- llama-3
- VRAM Q4
- 42 GB
NVIDIA's RLHF-tuned Llama 3.1 70B. Tops several Arena-style human-preference leaderboards and shipped with NVIDIA's reward-model research. Inherits the Llama 3 community licence.
- Context
- 128K
- License
- llama-3
- VRAM Q4
- 42 GB
R1 reasoning capabilities distilled into a Llama 3.3 70B base. The most accessible way to run R1-class reasoning locally — fits on a single H100 in fp16 or on a 4090 at Q4. Inherits Llama 3's community licence (commercial use under 700M MAU). Great pick for production reasoning workloads where the full R1 is too expensive to host but o1/R1-style quality is required.
- Context
- 128K
- License
- llama-3
- VRAM Q4
- 42 GB
Meta's December 2024 refresh of Llama 3 70B that closes most of the gap with Llama 3.1 405B for chat workloads while remaining tractable on a single H100. Strong instruction following, robust tool-use behaviour, and a 128K context window make it the default choice for production chat at 70B scale. The 3.3 release was trained on a refreshed instruction-tuning data mix and benefits from Meta's most recent alignment work. It outperforms the much larger 3.1 405B on several reasoning benchmarks at a fraction of inference cost. The licence is the Llama 3 Community License, which permits commercial use unless your service exceeds 700M monthly active users. Good pick for: production chat at scale, RAG over long documents, agentic workflows where tool use matters, and any 70B-tier replacement for closed proprietary models.
- Context
- 128K
- License
- llama-3
- VRAM Q4
- 42 GB
Cohere's 35B model tuned for RAG and tool use. The open weights are released under CC-BY-NC (commercial use requires the Cohere API). Strong multilingual coverage and a fine-grained RAG-mode output format that makes downstream citation easier.
- Context
- 128K
- License
- mrl
- VRAM Q4
- 21 GB
Coding-specialised Qwen2.5 32B fine-tune. GPT-4o-class on HumanEval and BigCodeBench at the time of release. Trained on additional code-heavy data with extended pre-training. Apache 2.0. Natural pick for self-hosted coding assistants, code-review automation, and any agent loop that primarily writes code.
- Context
- 128K
- License
- apache-2-0
- VRAM Q4
- 19.2 GB
32B sweet-spot model: strong reasoning, fits on one H100 in fp16, on a 4090 at Q4. The 32B size in particular hits a quality/cost knee — quality scales with parameters faster than cost up to ~32B, and slower afterwards. Favoured for production chat where 7B isn't sharp enough and where 70B+ would over-spec the hardware budget. Apache 2.0 licence.
- Context
- 128K
- License
- apache-2-0
- VRAM Q4
- 19.2 GB
Meta's April 2025 mixture-of-experts release. 17B active parameters across 16 experts (109B total). Natively multimodal with an unprecedented 10M-token context window — a leap far beyond Llama 3's 128K. Scout was designed to run on a single GPU at Q4 while beating Llama 3.3 70B on reasoning and multilingual benchmarks. The Llama 4 licence tightened acceptable-use provisions vs Llama 3.
- Context
- 10.0M
- License
- llama-4
- VRAM Q4
- 10.2 GB
Larger Llama 4 sibling of Scout — 17B active across 128 experts (400B total). 1M-token native context. Positioned as GPT-4o-class on chat and reasoning while remaining tractable on a single high-end host at fp8. Multimodal from the ground up; instruction-tuned by Meta with a heavier synthetic-data pipeline than Llama 3.
- Context
- 1.0M
- License
- llama-4
- VRAM Q4
- 10.2 GB
Phi-3's mid-tier model with extended 128K context. MIT licence. Strong reasoning relative to its parameter count thanks to Microsoft's heavy investment in synthetic training data.
- Context
- 128K
- License
- mit
- VRAM Q4
- 8.4 GB
Mid-size Qwen2.5 with broad task coverage. The sweet spot for users who want noticeably better quality than 7B but can't justify the hardware footprint of 32B or 72B.
- Context
- 128K
- License
- apache-2-0
- VRAM Q4
- 8.4 GB
Joint Mistral × NVIDIA model with 128K context, designed as a drop-in upgrade to Mistral 7B. Trained with NVIDIA's Megatron stack and released under Apache 2.0. Strong multilingual coverage thanks to the Tekken tokenizer.
- Context
- 128K
- License
- apache-2-0
- VRAM Q4
- 7.2 GB
Llama 3's first vision-language model. Image understanding via a separately-trained ViT adapter bolted onto Llama 3 weights. Useful for OCR-adjacent workloads, document understanding, and image captioning at a permissive licence. The 11B size makes it cheap to host. Combined with the 128K text context, it handles long PDF-with-images workflows comfortably on a single 4090.
- Context
- 128K
- License
- llama-3
- VRAM Q4
- 6.6 GB
NousResearch's community-driven fine-tune on the Llama 3.1 8B base. Tuned for strong tool use, function calling and steerable persona behaviour. Inherits Llama 3's community licence and its 128K context.
- Context
- 128K
- License
- llama-3
- VRAM Q4
- 4.8 GB
The workhorse 8B instruction-tuned model. Excellent quality-to-cost ratio and the broadest ecosystem support of any open-weights model — every major inference engine, fine-tuning library, and quantization toolchain has a 3.1 8B preset. Fits in 24 GB of VRAM at fp16, ~6 GB at Q4. Strong default for production chat where 70B is overkill, for fine-tuning on a specialist task, and for any workload where you want a known-good baseline.
- Context
- 128K
- License
- llama-3
- VRAM Q4
- 4.8 GB
Apache-2.0-licensed 7B model with surprisingly strong reasoning and multilingual chops. Qwen 2.5 trains on a larger and more carefully filtered corpus than the original Qwen series, and the 7B variant punches well above its weight on coding and math benchmarks. A strong default for cost-sensitive chat workloads and for fine-tuning experiments where the Apache licence simplifies downstream redistribution.
- Context
- 128K
- License
- apache-2-0
- VRAM Q4
- 4.2 GB
Pocket-sized Llama 3 variant for edge deployment. Surprising chat quality after instruction tuning makes it competitive with much larger models from a previous generation. At Q4 it fits in ~2 GB of VRAM and runs on consumer GPUs and recent Apple Silicon. A strong default for on-device chat, summarisation, and structured extraction tasks where the workload doesn't need frontier reasoning quality.
- Context
- 128K
- License
- llama-3
- VRAM Q4
- 1.8 GB
The smallest Llama 3 release, designed for on-device inference on phones and laptops. The 1B model runs comfortably in <2 GB of RAM at Q4 quantization and is fast enough for real-time chat on a modern smartphone. Useful for edge inference, on-device assistants where round-tripping to a server is undesirable, and as a draft model for speculative decoding in front of a larger Llama 3 variant.
- Context
- 128K
- License
- llama-3
- VRAM Q4
- 0.6 GB