<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom">
  <channel>
    <title>Prompt Caching &amp; KV Prefix Reuse — FoundationIndex</title>
    <link>https://foundationindex.ustunozgur.com/s/prompt-caching</link>
    <description>Saving the precomputed internal state of repeated prompt openings—such as system instructions, tool schemas, or reference documents—so an LLM server or API can skip re-reading them on subsequent requests.</description>
    <language>en</language>
    <atom:link href="https://foundationindex.ustunozgur.com/s/prompt-caching/feed.xml" rel="self" type="application/rss+xml" />
    <item>
      <title>Prompt Caching &amp; KV Prefix Reuse · Ecosystem · 2026-05-06</title>
      <link>https://foundationindex.ustunozgur.com/s/prompt-caching#prompt-caching-vllm-mooncake-nixl-2026</link>
      <guid>https://foundationindex.ustunozgur.com/s/prompt-caching#prompt-caching-vllm-mooncake-nixl-2026</guid>
      <pubDate>Wed, 06 May 2026 00:00:00 GMT</pubDate>
      <description>&lt;p&gt;The vLLM and Mooncake teams integrated Mooncake Store with vLLM&amp;apos;s KV connector interface using zero-copy GPUDirect RDMA transfers, achieving 3.8× higher throughput and up to 46× lower median time-to-first-token on multi-turn agentic workloads by pooling evicted prefix blocks across cluster nodes.&lt;/p&gt;&lt;p&gt;Sources: &lt;a href=&quot;https://vllm.ai/blog/2026-05-06-mooncake-store&quot;&gt;Serving Agentic Workloads at Scale with vLLM x Mooncake&lt;/a&gt;&lt;/p&gt;</description>
    </item>
    <item>
      <title>Prompt Caching &amp; KV Prefix Reuse · Paper · 2025-10-08</title>
      <link>https://foundationindex.ustunozgur.com/s/prompt-caching#prompt-caching-lmcache-paper-2025</link>
      <guid>https://foundationindex.ustunozgur.com/s/prompt-caching#prompt-caching-lmcache-paper-2025</guid>
      <pubDate>Wed, 08 Oct 2025 00:00:00 GMT</pubDate>
      <description>&lt;p&gt;Liu et al. published LMCache, an open-source distributed KV cache layer that decouples prefix extraction, multi-tier CPU/disk offloading, and peer-to-peer RDMA transfers from single-instance memory, integrating with vLLM to reuse non-prefix and cross-node context chunks.&lt;/p&gt;&lt;p&gt;Sources: &lt;a href=&quot;https://arxiv.org/abs/2510.09665&quot;&gt;LMCache: An Efficient KV Cache Layer for Enterprise-Scale LLM Inference&lt;/a&gt;&lt;/p&gt;</description>
    </item>
    <item>
      <title>Prompt Caching &amp; KV Prefix Reuse · Release · 2025-05-08</title>
      <link>https://foundationindex.ustunozgur.com/s/prompt-caching#prompt-caching-gemini-implicit-2025</link>
      <guid>https://foundationindex.ustunozgur.com/s/prompt-caching#prompt-caching-gemini-implicit-2025</guid>
      <pubDate>Thu, 08 May 2025 00:00:00 GMT</pubDate>
      <description>&lt;p&gt;Building on the explicit Context Caching API launched for Gemini 1.5 in May 2024, Google enabled implicit context caching by default on Gemini 2.5 models, automatically passing a 75% token discount whenever incoming requests share a prefix with cached contexts.&lt;/p&gt;&lt;p&gt;Sources: &lt;a href=&quot;https://developers.googleblog.com/en/gemini-2-5-models-now-support-implicit-caching/&quot;&gt;Gemini 2.5 Models now support implicit caching&lt;/a&gt;&lt;/p&gt;</description>
    </item>
    <item>
      <title>Prompt Caching &amp; KV Prefix Reuse · Release · 2024-10-01</title>
      <link>https://foundationindex.ustunozgur.com/s/prompt-caching#prompt-caching-openai-automatic-2024</link>
      <guid>https://foundationindex.ustunozgur.com/s/prompt-caching#prompt-caching-openai-automatic-2024</guid>
      <pubDate>Tue, 01 Oct 2024 00:00:00 GMT</pubDate>
      <description>&lt;p&gt;OpenAI rolled out automatic Prompt Caching across GPT-4o and o1 API endpoints, routing requests with matching prefixes longer than 1,024 tokens to cache-warm workers and applying an automatic 50% input-token discount without manual cache-control annotations.&lt;/p&gt;&lt;p&gt;Sources: &lt;a href=&quot;https://openai.com/index/api-prompt-caching/&quot;&gt;Prompt Caching in the API&lt;/a&gt;&lt;/p&gt;</description>
    </item>
    <item>
      <title>Prompt Caching &amp; KV Prefix Reuse · Release · 2024-08-14</title>
      <link>https://foundationindex.ustunozgur.com/s/prompt-caching#prompt-caching-anthropic-api-2024</link>
      <guid>https://foundationindex.ustunozgur.com/s/prompt-caching#prompt-caching-anthropic-api-2024</guid>
      <pubDate>Wed, 14 Aug 2024 00:00:00 GMT</pubDate>
      <description>&lt;p&gt;Anthropic launched Prompt Caching on the Claude API, letting developers mark reusable prefix breakpoints on system instructions, tool definitions, and large documents to cut input token costs by up to 90% and time-to-first-token latency by up to 85%.&lt;/p&gt;&lt;p&gt;Sources: &lt;a href=&quot;https://www.anthropic.com/news/prompt-caching&quot;&gt;Prompt caching with Claude&lt;/a&gt;&lt;/p&gt;</description>
    </item>
    <item>
      <title>Prompt Caching &amp; KV Prefix Reuse · Paper · 2024-06-24</title>
      <link>https://foundationindex.ustunozgur.com/s/prompt-caching#prompt-caching-mooncake-paper-2024</link>
      <guid>https://foundationindex.ustunozgur.com/s/prompt-caching#prompt-caching-mooncake-paper-2024</guid>
      <pubDate>Mon, 24 Jun 2024 00:00:00 GMT</pubDate>
      <description>&lt;p&gt;Qin et al. at Moonshot AI and Tsinghua University published Mooncake, introducing a KV-cache-centric serving architecture that pools prefix caches across CPU DRAM and SSD storage and uses cache-aware scheduling to boost long-context throughput in production Kimi deployments.&lt;/p&gt;&lt;p&gt;Sources: &lt;a href=&quot;https://arxiv.org/abs/2407.00079&quot;&gt;Mooncake: A KVCache-centric Disaggregated Architecture for LLM Serving&lt;/a&gt;&lt;/p&gt;</description>
    </item>
  </channel>
</rss>
