IndexMapAbout
ArchitectureLLMCross-Modal

State Space Models & Mamba

Also known as: Mamba · Mamba-2 · State Space Models · SSM · Jamba

Plain English

An alternative neural network architecture to Transformer attention that processes long sequences in linear time with a fixed-size memory state—avoiding the quadratic slowdown and growing KV cache of standard Transformers.

Technical Overview

Selective State Space Models (Mamba) parameterize continuous-time state transitions as input-dependent recurrences computed via hardware-aware parallel scans on GPUs. Mamba-2 bridged SSMs and linear attention via Structured State Space Duality (SSD), while hybrid architectures like Jamba and Qwen3.5 interleave SSM layers with self-attention and MoE blocks to combine constant-memory long context with sharp in-context recall.

Timeline (3)

RSS

2021

2023

2024

See also (bkz.) (5)

Open full graph