Snell et al. published Scaling LLM Test-Time Compute Optimally, demonstrating that adaptive inference-time search and revision guided by process reward models can beat a 14× larger model on math reasoning under matched compute budgets.
Reasoning & Test-Time Compute
Also known as: Test-Time Compute · Inference-Time Scaling · Reasoning Models · Chain-of-Thought RL
Plain English
Training AI models to spend extra thinking time before answering—generating, verifying, and refining intermediate reasoning steps at inference time rather than blurting out the first token.
Technical Overview
Test-time compute scaling shifts performance gains from pre-training FLOPs alone to inference-time deliberation via reinforcement learning (GRPO/PPO) over long chain-of-thought traces, process reward models (PRMs), and best-of-N search. Pioneered by OpenAI o1 and open-sourced by DeepSeek-R1, reasoning models trade higher latency and token output for steep accuracy gains on math, code, and planning.
Timeline (4)
2024
- Paper
- Release
OpenAI launched o1-preview, establishing that large-scale reinforcement learning over private chain-of-thought tokens produces smooth performance scaling with both RL training compute and inference-time thinking tokens.
2025
- Paper
DeepSeek published DeepSeek-R1, showing that pure reinforcement learning with verifiable rewards (GRPO) spontaneously elicits self-reflection, backtracking, and long-horizon reasoning without human-written reasoning traces, while open-sourcing distilled reasoning Checkpoints.
- Release
Google DeepMind launched Gemini 2.5 Pro as a thinking-by-default frontier model, merging test-time reasoning with a 1M-token native multimodal context window—signaling industry convergence between general-purpose and reasoning models.
See also (bkz.) (14)
Open full graphReasoning & Test-Time Compute