FIFoundationIndex
IndexMapAbout
Inspect on Graph
ConceptLLM

Prefill/Decode Disaggregation

Also known as: Disaggregated serving · P/D disaggregation · Phase splitting

Plain English

One group of machines reads the prompt, then hands the request to another group that generates the reply token by token. An operator can add capacity to whichever group is making users wait.

Technical Overview

Prefill computes the prompt's key and value tensors. Moving that state to decode workers lets an operator choose hardware and parallelism separately for each phase. The transfer adds latency that depends on the cache size and the bandwidth between workers.

Chronological Entries (2)

Direct Relations (1)