IndexMapAbout
ArchitectureLLMCross-Modal

PagedAttention

Also known as: Paged KV Cache

Plain English

A GPU memory technique borrowed from operating systems that splits an LLM's short-term conversation memory into fixed-size pages so servers stop wasting VRAM and can serve far more users simultaneously.

Technical Overview

Traditional serving engines pre-allocated contiguous GPU memory buffers for each sequence's maximum KV cache length, wasting 60–80% of VRAM to fragmentation and over-reservation. PagedAttention partitions each request's KV cache into fixed-size non-contiguous physical blocks tracked by a logical-to-physical block table, enabling near-zero waste, copy-on-write prefix sharing, and higher batch concurrency inside vLLM and other Inference Engines.

Chronological Entries (2)

Direct Relations (4)