Google researchers introduced FLAN, fine-tuning a 137B-parameter model on more than 60 NLP datasets rewritten as natural-language instructions. The tuned model beat zero-shot GPT-3 175B on 20 of 25 evaluated tasks, showing that instruction-formatted supervision transfers to tasks never seen in tuning.
Supervised Fine-Tuning (SFT)
Also known as: SFT · Instruction Tuning
Plain English
Continuing to train a pre-trained model on curated examples of the desired behavior, such as questions paired with good answers, so it follows instructions instead of just continuing text.
Technical Overview
SFT minimizes cross-entropy on target responses given prompts, usually with loss masked to the response tokens, starting from a base checkpoint. It is cheap relative to pre-training and is the usual first stage of post-training, before preference optimization such as RLHF or DPO. Quality and diversity of examples matter more than count, and demonstrations can come from humans, from synthetic data, or from a stronger model's reasoning traces. Pure imitation can limit generalization compared with reinforcement learning on outcomes.
Timeline (7)
2021
Paper2022
PaperSelf-Instruct had a model generate its own instructions, inputs, and outputs from a small seed set, then filtered them and fine-tuned on the result. Applied to vanilla GPT-3, it gave a 33% absolute gain on Super-NaturalInstructions, which made synthetic instruction sets a cheap substitute for human annotation.
2023
PaperLIMA fine-tuned a 65B LLaMA model on only 1,000 carefully curated prompt-response pairs, with no reinforcement learning. Its authors argued that most knowledge comes from pre-training and that SFT mainly teaches response style, so example quality outweighs quantity.
2024
PaperThe Allen Institute for AI published Tulu 3, a fully documented post-training recipe for Llama 3.1 base models that starts with an SFT stage on a mix of public and synthetic prompts, then applies DPO and verifiable-reward reinforcement learning. The paper released data mixes, code, and ablations, so teams could reproduce the SFT stage rather than guess at it.
2025
PaperA comparative study found that on rule-based text and visual tasks, outcome-reward reinforcement learning generalized to unseen rule variants while SFT tended to memorize training examples. It also found SFT still useful for stabilizing output format before RL, which is why the two stages are often combined instead of treated as rivals.
- Paper
The s1 paper fine-tuned Qwen2.5-32B-Instruct on just 1,000 curated question-and-reasoning-trace pairs (s1K), then used a decoding-time budget-forcing control to lengthen or cut thinking. It showed that a small SFT set of reasoning traces can give a model competitive test-time scaling behavior.
2026
PaperThe OpenThoughts-Agent project systematically ablated data curation for multi-turn agentic SFT and RL across terminal and software-engineering tasks, finding that the choice and diversity of task sources affected how well models generalized across agent benchmarks.
See also (14)
Open full graphSupervised Fine-Tuning (SFT)
9 more connections
- Distillation & Synthetic DataIncoming · Uses
- LoRA & QLoRAIncoming · Part of
- Vision-Language-Action ModelsIncoming · Uses
- Agent HarnessOutgoing · Mentions
- Allen Institute for AIIncoming · Mentions
- GPTIncoming · Mentions
- LlamaOutgoing · Mentions
- Pre-TrainingOutgoing · Mentions
- Reasoning & Test-Time ComputeOutgoing · MentionsIncoming · Mentions