<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom">
  <channel>
    <title>Model Merging — FoundationIndex</title>
    <link>https://foundationindex.ustunozgur.com/s/model-merging</link>
    <description>Combining the weights of two or more fine-tuned AI models into a single checkpoint without running any new training—blending skills like coding, math, and chat for free.</description>
    <language>en</language>
    <atom:link href="https://foundationindex.ustunozgur.com/s/model-merging/feed.xml" rel="self" type="application/rss+xml" />
    <item>
      <title>Model Merging · Paper · 2024-03-20</title>
      <link>https://foundationindex.ustunozgur.com/s/model-merging#model-merging-mergekit-2024</link>
      <guid>https://foundationindex.ustunozgur.com/s/model-merging#model-merging-mergekit-2024</guid>
      <pubDate>Wed, 20 Mar 2024 00:00:00 GMT</pubDate>
      <description>&lt;p&gt;Goddard et al. published MergeKit, an open-source toolkit implementing SLERP, TIES, DARE, and Frankenmerging that powered a wave of community-merged open-weight models on Hugging Face.&lt;/p&gt;&lt;p&gt;Sources: &lt;a href=&quot;https://arxiv.org/abs/2403.13257&quot;&gt;Arcee&amp;apos;s MergeKit: A Toolkit for Merging Large Language Models&lt;/a&gt;&lt;/p&gt;</description>
    </item>
    <item>
      <title>Model Merging · Paper · 2023-06-02</title>
      <link>https://foundationindex.ustunozgur.com/s/model-merging#model-merging-ties-2023</link>
      <guid>https://foundationindex.ustunozgur.com/s/model-merging#model-merging-ties-2023</guid>
      <pubDate>Fri, 02 Jun 2023 00:00:00 GMT</pubDate>
      <description>&lt;p&gt;Yadav et al. proposed TIES-Merging, resolving sign conflicts and redundant parameter magnitudes across task vectors before averaging—preventing destructive interference when combining three or more specialized Checkpoints.&lt;/p&gt;&lt;p&gt;Sources: &lt;a href=&quot;https://arxiv.org/abs/2306.01708&quot;&gt;TIES-Merging: Resolving Interference When Merging Models&lt;/a&gt;&lt;/p&gt;</description>
    </item>
    <item>
      <title>Model Merging · Paper · 2022-12-08</title>
      <link>https://foundationindex.ustunozgur.com/s/model-merging#model-merging-task-arithmetic-2022</link>
      <guid>https://foundationindex.ustunozgur.com/s/model-merging#model-merging-task-arithmetic-2022</guid>
      <pubDate>Thu, 08 Dec 2022 00:00:00 GMT</pubDate>
      <description>&lt;p&gt;Ilharco et al. introduced Task Arithmetic, defining task vectors (θ_ft - θ_base) that can be added to compose capabilities or subtracted to unlearn unwanted behaviors without gradient updates.&lt;/p&gt;&lt;p&gt;Sources: &lt;a href=&quot;https://arxiv.org/abs/2212.04089&quot;&gt;Editing Models with Task Arithmetic&lt;/a&gt;&lt;/p&gt;</description>
    </item>
    <item>
      <title>Model Merging · Paper · 2022-03-10</title>
      <link>https://foundationindex.ustunozgur.com/s/model-merging#model-merging-soups-2022</link>
      <guid>https://foundationindex.ustunozgur.com/s/model-merging#model-merging-soups-2022</guid>
      <pubDate>Thu, 10 Mar 2022 00:00:00 GMT</pubDate>
      <description>&lt;p&gt;Wortsman et al. published Model Soups, showing that averaging the weights of multiple fine-tuned runs from the same pre-trained checkpoint improves accuracy and robustness with zero extra inference cost.&lt;/p&gt;&lt;p&gt;Sources: &lt;a href=&quot;https://arxiv.org/abs/2203.05482&quot;&gt;Model Soups: Averaging Weights of Multiple Fine-tuned Models Improves Accuracy without Increasing Inference Time&lt;/a&gt;&lt;/p&gt;</description>
    </item>
  </channel>
</rss>
