AI21 Labs
Jamba
Hybrid Mamba/Transformer architecture with very long native context windows.
Visit homepage ↗History & context
AI21 Labs' Jamba (March 2024) was the first major hybrid Mamba/Transformer/MoE model from a frontier lab. The state-space-model layers give Jamba linear-time scaling with sequence length, which makes very-long-context workloads more practical than with pure-attention architectures.
Jamba 1.5 Large (August 2024) is the current flagship — 398B total parameters with 94B active, native 256K context (effective beyond 140K). Licensed under AI21's open model licence, which permits most commercial use.