<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom">
  <channel>
    <title>FlashInfer — FoundationIndex</title>
    <link>https://foundationindex.ustunozgur.com/s/flashinfer</link>
    <description>An open-source GPU kernel library for LLM serving that computes attention, sampling, and Mixture-of-Experts layers over paged and prefix-shared memory without wasting bandwidth.</description>
    <language>en</language>
    <atom:link href="https://foundationindex.ustunozgur.com/s/flashinfer/feed.xml" rel="self" type="application/rss+xml" />
    <item>
      <title>FlashInfer · Release · 2026-09-22</title>
      <link>https://foundationindex.ustunozgur.com/s/flashinfer#flashinfer-v07-megamoe-2026</link>
      <guid>https://foundationindex.ustunozgur.com/s/flashinfer#flashinfer-v07-megamoe-2026</guid>
      <pubDate>Tue, 22 Sep 2026 00:00:00 GMT</pubDate>
      <description>&lt;p&gt;FlashInfer v0.7 expanded the library beyond attention and GEMMs with MegaMoE fused expert-parallel MoE kernels that overlap NVLink communication with low-precision FP8/NVFP4 computation, alongside Autotuner v2 for serving-aware runtime kernel selection and open CUTLASS/CuTe-DSL kernel paths.&lt;/p&gt;&lt;p&gt;Sources: &lt;a href=&quot;https://flashinfer.ai/2026/09/22/flashinfer-v07.html&quot;&gt;Accelerate LLM Inference with Open Kernels and Smarter Autotuning with FlashInfer v0.7&lt;/a&gt;, &lt;a href=&quot;https://flashinfer.ai/2026/09/22/mega-moe.html&quot;&gt;MegaMoE in FlashInfer: Fused Expert-Parallel MoE Kernels&lt;/a&gt;&lt;/p&gt;</description>
    </item>
    <item>
      <title>FlashInfer · Paper · 2026-01-01</title>
      <link>https://foundationindex.ustunozgur.com/s/flashinfer#flashinfer-bench-paper-2026</link>
      <guid>https://foundationindex.ustunozgur.com/s/flashinfer#flashinfer-bench-paper-2026</guid>
      <pubDate>Thu, 01 Jan 2026 00:00:00 GMT</pubDate>
      <description>&lt;p&gt;Xing et al. published FlashInfer-Bench, defining a standardized trace schema (FlashInfer Trace), real-world serving workloads, and a dynamic apply() substitution mechanism that lets autonomous coding agents generate, benchmark, and hot-swap low-level GPU kernels directly inside production vLLM and SGLang engines.&lt;/p&gt;&lt;p&gt;Sources: &lt;a href=&quot;https://arxiv.org/abs/2601.00227&quot;&gt;FlashInfer-Bench: Building the Virtuous Cycle for AI-driven LLM Systems&lt;/a&gt;&lt;/p&gt;</description>
    </item>
    <item>
      <title>FlashInfer · Milestone · 2025-03-10</title>
      <link>https://foundationindex.ustunozgur.com/s/flashinfer#flashinfer-sorting-free-sampling-2025</link>
      <guid>https://foundationindex.ustunozgur.com/s/flashinfer#flashinfer-sorting-free-sampling-2025</guid>
      <pubDate>Mon, 10 Mar 2025 00:00:00 GMT</pubDate>
      <description>&lt;p&gt;FlashInfer introduced sorting-free GPU sampling kernels for Top-K, Top-P, and Min-P token sampling, replacing full vocabulary radix sorts with dual-pivot rejection sampling and segmented prefix scans so CPU-free sampling could be captured directly inside CUDA Graphs.&lt;/p&gt;&lt;p&gt;Sources: &lt;a href=&quot;https://flashinfer.ai/2025/03/10/sampling.html&quot;&gt;Sorting-Free GPU Kernels for LLM Sampling&lt;/a&gt;&lt;/p&gt;</description>
    </item>
    <item>
      <title>FlashInfer · Paper · 2025-01-02</title>
      <link>https://foundationindex.ustunozgur.com/s/flashinfer#flashinfer-mlsys-paper-2025</link>
      <guid>https://foundationindex.ustunozgur.com/s/flashinfer#flashinfer-mlsys-paper-2025</guid>
      <pubDate>Thu, 02 Jan 2025 00:00:00 GMT</pubDate>
      <description>&lt;p&gt;Ye et al. published the FlashInfer paper (accepted to MLSys 2025), unifying ragged, paged, and radix-tree KV caches under a block-sparse matrix representation with load-balanced thread-block scheduling. Compared to compiler-generated backends, FlashInfer reduced inter-token latency by 29–69% on standard serving benchmarks and 13–17% on long-context generation.&lt;/p&gt;&lt;p&gt;Sources: &lt;a href=&quot;https://arxiv.org/abs/2501.01005&quot;&gt;FlashInfer: Efficient and Customizable Attention Engine for LLM Inference Serving&lt;/a&gt;&lt;/p&gt;</description>
    </item>
    <item>
      <title>FlashInfer · Release · 2024-12-16</title>
      <link>https://foundationindex.ustunozgur.com/s/flashinfer#flashinfer-v02-jit-release-2024</link>
      <guid>https://foundationindex.ustunozgur.com/s/flashinfer#flashinfer-v02-jit-release-2024</guid>
      <pubDate>Mon, 16 Dec 2024 00:00:00 GMT</pubDate>
      <description>&lt;p&gt;The FlashInfer team released FlashInfer v0.2, introducing a PyTorch-based JIT compilation engine for custom attention variants (such as logit soft-capping and RoPE fusion), deterministic plan/run scheduling for CUDA Graph capture, and native DeepSeek Multi-head Latent Attention (MLA) support in vLLM and SGLang.&lt;/p&gt;&lt;p&gt;Sources: &lt;a href=&quot;https://flashinfer.ai/2024/12/16/flashinfer-v02-release.html&quot;&gt;FlashInfer 0.2 - Efficient and Customizable Kernels for LLM Inference Serving&lt;/a&gt;&lt;/p&gt;</description>
    </item>
    <item>
      <title>FlashInfer · Release · 2024-02-02</title>
      <link>https://foundationindex.ustunozgur.com/s/flashinfer#flashinfer-cascade-launch-2024</link>
      <guid>https://foundationindex.ustunozgur.com/s/flashinfer#flashinfer-cascade-launch-2024</guid>
      <pubDate>Fri, 02 Feb 2024 00:00:00 GMT</pubDate>
      <description>&lt;p&gt;Researchers at the University of Washington, OctoAI, and CMU launched FlashInfer alongside Cascade Inference, decoupling shared-prefix attention from unique per-request suffixes. Keeping shared KV cache tiles in GPU shared memory (SMEM) while merging partial outputs via log-sum-exp states cut redundant HBM reads and accelerated shared-prefix batch decoding by up to 31× on H100 GPUs.&lt;/p&gt;&lt;p&gt;Sources: &lt;a href=&quot;https://flashinfer.ai/2024/02/02/cascade-inference.html&quot;&gt;Cascade Inference: Memory Bandwidth Efficient Shared Prefix Batch Decoding&lt;/a&gt;, &lt;a href=&quot;https://flashinfer.ai/2024/02/02/introduce-flashinfer.html&quot;&gt;Accelerating Self-Attentions for LLM Serving with FlashInfer&lt;/a&gt;&lt;/p&gt;</description>
    </item>
  </channel>
</rss>
