IndexMapAbout
Tool / RuntimeLLMCross-Modal

SGLang

Also known as: SGLang Runtime · SGLang Server

Plain English

SGLang is a runtime for serving AI models. It can reuse work when requests share the same opening text, such as a long instruction or document.

Technical Overview

RadixAttention indexes reusable KV cache state by prompt prefix. SGLang also supports disaggregated serving and expert parallelism, which let deployments allocate hardware differently for prompt processing and token generation.

Chronological Entries (3)

Direct Relations (2)