<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom">
  <channel>
    <title>State Space Models &amp; Mamba — FoundationIndex</title>
    <link>https://foundationindex.ustunozgur.com/s/mamba</link>
    <description>An alternative neural network architecture to Transformer attention that processes long sequences in linear time with a fixed-size memory state—avoiding the quadratic slowdown and growing KV cache of standard Transformers.</description>
    <language>en</language>
    <atom:link href="https://foundationindex.ustunozgur.com/s/mamba/feed.xml" rel="self" type="application/rss+xml" />
    <item>
      <title>State Space Models &amp; Mamba · Paper · 2024-05-31</title>
      <link>https://foundationindex.ustunozgur.com/s/mamba#mamba-2-ssd-2024</link>
      <guid>https://foundationindex.ustunozgur.com/s/mamba#mamba-2-ssd-2024</guid>
      <pubDate>Fri, 31 May 2024 00:00:00 GMT</pubDate>
      <description>&lt;p&gt;Dao and Gu published Mamba-2, formalizing Structured State Space Duality (SSD) to unify selective SSMs with linear attention and enabling Tensor Core matrix-multiplication speedups 2–8× faster than Mamba-1.&lt;/p&gt;&lt;p&gt;Sources: &lt;a href=&quot;https://arxiv.org/abs/2405.21060&quot;&gt;Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Duality&lt;/a&gt;&lt;/p&gt;</description>
    </item>
    <item>
      <title>State Space Models &amp; Mamba · Paper · 2023-12-01</title>
      <link>https://foundationindex.ustunozgur.com/s/mamba#mamba-selective-ssm-2023</link>
      <guid>https://foundationindex.ustunozgur.com/s/mamba#mamba-selective-ssm-2023</guid>
      <pubDate>Fri, 01 Dec 2023 00:00:00 GMT</pubDate>
      <description>&lt;p&gt;Gu and Dao introduced Mamba, making SSM state-transition parameters input-selective and writing a fused SRAM parallel-scan kernel—achieving 5× higher inference throughput than Transformers at linear O(N) sequence scaling.&lt;/p&gt;&lt;p&gt;Sources: &lt;a href=&quot;https://arxiv.org/abs/2312.00752&quot;&gt;Mamba: Linear-Time Sequence Modeling with Selective State Spaces&lt;/a&gt;&lt;/p&gt;</description>
    </item>
    <item>
      <title>State Space Models &amp; Mamba · Paper · 2021-10-31</title>
      <link>https://foundationindex.ustunozgur.com/s/mamba#mamba-s4-2021</link>
      <guid>https://foundationindex.ustunozgur.com/s/mamba#mamba-s4-2021</guid>
      <pubDate>Sun, 31 Oct 2021 00:00:00 GMT</pubDate>
      <description>&lt;p&gt;Gu, Goel, and Ré published S4 (Structured State Spaces for Sequence Modeling), showing that HiPPO-initialized linear state-space layers could model 16K+ long-range sequences without quadratic attention complexity.&lt;/p&gt;&lt;p&gt;Sources: &lt;a href=&quot;https://arxiv.org/abs/2111.00396&quot;&gt;Efficiently Modeling Long Sequences with Structured State Spaces&lt;/a&gt;&lt;/p&gt;</description>
    </item>
  </channel>
</rss>
