IndexMapAnti-DoomscrollAbout
See also
ConceptLLMCross-Modal

Supervised Fine-Tuning (SFT)

Also known as: SFT · Instruction Tuning

Plain English

Continuing to train a pre-trained model on curated examples of the desired behavior, such as questions paired with good answers, so it follows instructions instead of just continuing text.

Technical Overview

SFT minimizes cross-entropy on target responses given prompts, usually with loss masked to the response tokens, starting from a base checkpoint. It is cheap relative to pre-training and is the usual first stage of post-training, before preference optimization such as RLHF or DPO. Quality and diversity of examples matter more than count, and demonstrations can come from humans, from synthetic data, or from a stronger model's reasoning traces. Pure imitation can limit generalization compared with reinforcement learning on outcomes.

Timeline (7)

RSS
  1. 2021

    Paper

    Google researchers introduced FLAN, fine-tuning a 137B-parameter model on more than 60 NLP datasets rewritten as natural-language instructions. The tuned model beat zero-shot GPT-3 175B on 20 of 25 evaluated tasks, showing that instruction-formatted supervision transfers to tasks never seen in tuning.

  2. 2022

  3. 2023

    Paper

    LIMA fine-tuned a 65B LLaMA model on only 1,000 carefully curated prompt-response pairs, with no reinforcement learning. Its authors argued that most knowledge comes from pre-training and that SFT mainly teaches response style, so example quality outweighs quantity.

  4. 2024

    Paper

    The Allen Institute for AI published Tulu 3, a fully documented post-training recipe for Llama 3.1 base models that starts with an SFT stage on a mix of public and synthetic prompts, then applies DPO and verifiable-reward reinforcement learning. The paper released data mixes, code, and ablations, so teams could reproduce the SFT stage rather than guess at it.

  5. 2025

  6. Paper

    The s1 paper fine-tuned Qwen2.5-32B-Instruct on just 1,000 curated question-and-reasoning-trace pairs (s1K), then used a decoding-time budget-forcing control to lengthen or cut thinking. It showed that a small SFT set of reasoning traces can give a model competitive test-time scaling behavior.

  7. 2026

See also (14)

Open full graph
9 more connections