IndexMapAbout
ConceptLLMCross-Modal

GRPO

Also known as: Group Relative Policy Optimization · RLVR

Plain English

A reinforcement learning method that grades a group of candidate answers from the same prompt against each other—often using automatic math or code checks—so models can learn step-by-step reasoning without needing a separate critic network.

Technical Overview

Introduced in DeepSeekMath, Group Relative Policy Optimization drops the learned value (critic) network required by PPO in RLHF and instead estimates advantages by normalizing rewards across a group of sampled completions for each prompt. Paired with rule-based verifiable rewards (RLVR) such as unit tests or math verifiers, GRPO halves RL memory overhead and became the primary algorithm behind open-weight reasoning Post-Training (SFT, RLHF, DPO & GRPO).

Chronological Entries (3)

#3Paper
AI-Assisted

ByteDance Seed and Tsinghua researchers published DAPO, an open-source large-scale RL recipe refining GRPO with asymmetric clip ranges, dynamic sampling, and token-level policy gradients to prevent entropy collapse and length bias during long-horizon reasoning runs.

Direct Relations (4)