<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom">
  <channel>
    <title>Supervised Fine-Tuning (SFT) — FoundationIndex</title>
    <link>https://foundationindex.ustunozgur.com/s/supervised-fine-tuning</link>
    <description>Continuing to train a pre-trained model on curated examples of the desired behavior, such as questions paired with good answers, so it follows instructions instead of just continuing text.</description>
    <language>en</language>
    <atom:link href="https://foundationindex.ustunozgur.com/s/supervised-fine-tuning/feed.xml" rel="self" type="application/rss+xml" />
    <item>
      <title>Supervised Fine-Tuning (SFT) · Paper · 2026-06-23</title>
      <link>https://foundationindex.ustunozgur.com/s/supervised-fine-tuning#sft-openthoughts-agent-2026</link>
      <guid>https://foundationindex.ustunozgur.com/s/supervised-fine-tuning#sft-openthoughts-agent-2026</guid>
      <pubDate>Tue, 23 Jun 2026 00:00:00 GMT</pubDate>
      <description>&lt;p&gt;The OpenThoughts-Agent project systematically ablated data curation for multi-turn agentic SFT and RL across terminal and software-engineering tasks, finding that the choice and diversity of task sources affected how well models generalized across agent benchmarks.&lt;/p&gt;&lt;p&gt;Sources: &lt;a href=&quot;https://arxiv.org/abs/2606.24855&quot;&gt;OpenThoughts-Agent: Data Recipes for Agentic Models&lt;/a&gt;&lt;/p&gt;</description>
    </item>
    <item>
      <title>Supervised Fine-Tuning (SFT) · Paper · 2025-01-31</title>
      <link>https://foundationindex.ustunozgur.com/s/supervised-fine-tuning#sft-s1-reasoning-traces-2025</link>
      <guid>https://foundationindex.ustunozgur.com/s/supervised-fine-tuning#sft-s1-reasoning-traces-2025</guid>
      <pubDate>Fri, 31 Jan 2025 00:00:00 GMT</pubDate>
      <description>&lt;p&gt;The s1 paper fine-tuned Qwen2.5-32B-Instruct on just 1,000 curated question-and-reasoning-trace pairs (s1K), then used a decoding-time budget-forcing control to lengthen or cut thinking. It showed that a small SFT set of reasoning traces can give a model competitive test-time scaling behavior.&lt;/p&gt;&lt;p&gt;Sources: &lt;a href=&quot;https://arxiv.org/abs/2501.19393&quot;&gt;s1: Simple test-time scaling&lt;/a&gt;&lt;/p&gt;</description>
    </item>
    <item>
      <title>Supervised Fine-Tuning (SFT) · Paper · 2025-01-28</title>
      <link>https://foundationindex.ustunozgur.com/s/supervised-fine-tuning#sft-memorizes-rl-generalizes-2025</link>
      <guid>https://foundationindex.ustunozgur.com/s/supervised-fine-tuning#sft-memorizes-rl-generalizes-2025</guid>
      <pubDate>Tue, 28 Jan 2025 00:00:00 GMT</pubDate>
      <description>&lt;p&gt;A comparative study found that on rule-based text and visual tasks, outcome-reward reinforcement learning generalized to unseen rule variants while SFT tended to memorize training examples. It also found SFT still useful for stabilizing output format before RL, which is why the two stages are often combined instead of treated as rivals.&lt;/p&gt;&lt;p&gt;Sources: &lt;a href=&quot;https://arxiv.org/abs/2501.17161&quot;&gt;SFT Memorizes, RL Generalizes: A Comparative Study of Foundation Model Post-training&lt;/a&gt;&lt;/p&gt;</description>
    </item>
    <item>
      <title>Supervised Fine-Tuning (SFT) · Paper · 2024-11-22</title>
      <link>https://foundationindex.ustunozgur.com/s/supervised-fine-tuning#sft-tulu3-2024</link>
      <guid>https://foundationindex.ustunozgur.com/s/supervised-fine-tuning#sft-tulu3-2024</guid>
      <pubDate>Fri, 22 Nov 2024 00:00:00 GMT</pubDate>
      <description>&lt;p&gt;The Allen Institute for AI published Tulu 3, a fully documented post-training recipe for Llama 3.1 base models that starts with an SFT stage on a mix of public and synthetic prompts, then applies DPO and verifiable-reward reinforcement learning. The paper released data mixes, code, and ablations, so teams could reproduce the SFT stage rather than guess at it.&lt;/p&gt;&lt;p&gt;Sources: &lt;a href=&quot;https://arxiv.org/abs/2411.15124&quot;&gt;Tulu 3: Pushing Frontiers in Open Language Model Post-Training&lt;/a&gt;&lt;/p&gt;</description>
    </item>
    <item>
      <title>Supervised Fine-Tuning (SFT) · Paper · 2023-05-18</title>
      <link>https://foundationindex.ustunozgur.com/s/supervised-fine-tuning#sft-lima-2023</link>
      <guid>https://foundationindex.ustunozgur.com/s/supervised-fine-tuning#sft-lima-2023</guid>
      <pubDate>Thu, 18 May 2023 00:00:00 GMT</pubDate>
      <description>&lt;p&gt;LIMA fine-tuned a 65B LLaMA model on only 1,000 carefully curated prompt-response pairs, with no reinforcement learning. Its authors argued that most knowledge comes from pre-training and that SFT mainly teaches response style, so example quality outweighs quantity.&lt;/p&gt;&lt;p&gt;Sources: &lt;a href=&quot;https://arxiv.org/abs/2305.11206&quot;&gt;LIMA: Less Is More for Alignment&lt;/a&gt;&lt;/p&gt;</description>
    </item>
    <item>
      <title>Supervised Fine-Tuning (SFT) · Paper · 2022-12-20</title>
      <link>https://foundationindex.ustunozgur.com/s/supervised-fine-tuning#sft-self-instruct-2022</link>
      <guid>https://foundationindex.ustunozgur.com/s/supervised-fine-tuning#sft-self-instruct-2022</guid>
      <pubDate>Tue, 20 Dec 2022 00:00:00 GMT</pubDate>
      <description>&lt;p&gt;Self-Instruct had a model generate its own instructions, inputs, and outputs from a small seed set, then filtered them and fine-tuned on the result. Applied to vanilla GPT-3, it gave a 33% absolute gain on Super-NaturalInstructions, which made synthetic instruction sets a cheap substitute for human annotation.&lt;/p&gt;&lt;p&gt;Sources: &lt;a href=&quot;https://arxiv.org/abs/2212.10560&quot;&gt;Self-Instruct: Aligning Language Models with Self-Generated Instructions&lt;/a&gt;&lt;/p&gt;</description>
    </item>
    <item>
      <title>Supervised Fine-Tuning (SFT) · Paper · 2021-09-03</title>
      <link>https://foundationindex.ustunozgur.com/s/supervised-fine-tuning#sft-flan-2021</link>
      <guid>https://foundationindex.ustunozgur.com/s/supervised-fine-tuning#sft-flan-2021</guid>
      <pubDate>Fri, 03 Sep 2021 00:00:00 GMT</pubDate>
      <description>&lt;p&gt;Google researchers introduced FLAN, fine-tuning a 137B-parameter model on more than 60 NLP datasets rewritten as natural-language instructions. The tuned model beat zero-shot GPT-3 175B on 20 of 25 evaluated tasks, showing that instruction-formatted supervision transfers to tasks never seen in tuning.&lt;/p&gt;&lt;p&gt;Sources: &lt;a href=&quot;https://arxiv.org/abs/2109.01652&quot;&gt;Finetuned Language Models Are Zero-Shot Learners&lt;/a&gt;&lt;/p&gt;</description>
    </item>
  </channel>
</rss>
