Willard and Louf published Efficient Guided Generation for Large Language Models (Outlines), recasting regex- and grammar-constrained decoding as finite-state machine transitions over pre-indexed vocabulary masks—turning O(V) per-step regex checks into O(1) lookups.
Structured Outputs & Constrained Decoding
Also known as: Structured Outputs · Constrained Decoding · JSON Schema Mode · XGrammar · Outlines
Plain English
Forcing an LLM to reply in an exact format like valid JSON, SQL, or a typed schema so downstream code never crashes on malformed model output.
Technical Overview
Constrained decoding compiles a regular expression, JSON Schema, or Context-Free Grammar (CFG) into a finite-state machine or pushdown automaton and masks invalid token logits at every decoding step. Libraries like Outlines and XGrammar—alongside native API strict: true modes—achieve near-zero latency overhead inside vLLM, SGLang, and TensorRT-LLM.
Timeline (3)
2023
- Paper
2024
- Release
OpenAI launched Structured Outputs in the API (
strict: true), combining context-free grammar constrained decoding with targeted fine-tuning sogpt-4o-2024-08-06achieved 100% adherence to developer-supplied JSON Schemas. - Paper
Dong et al. introduced XGrammar, splitting token vocabularies into context-independent precomputed masks and context-dependent pushdown checks—cutting CFG-constrained decoding overhead to near zero inside vLLM, SGLang, and TensorRT-LLM.
See also (bkz.) (7)
Open full graphStructured Outputs & Constrained Decoding