IndexMapAnti-DoomscrollAbout
ConceptLLMImageCross-Modal

Pruning and Sparsity

Also known as: model pruning · structured sparsity · 2:4 sparsity · weight pruning

Plain English

Making a trained model smaller or faster by removing weights, neurons, layers, or experts that matter least. The pruned model usually needs a short round of retraining to recover quality.

Technical Overview

Unstructured pruning zeroes individual weights, which saves storage but speeds up inference only with sparse kernels. Semi-structured patterns such as NVIDIA's 2:4 (two nonzeros in every four values) map to Sparse Tensor Cores, while structured pruning removes whole heads, channels, layers, or MoE experts for speedups on any hardware. One-shot LLM methods like SparseGPT and Wanda prune without retraining, whereas Minitron-style pipelines prune then recover accuracy with Distillation & Synthetic Data. Pruning composes with Quantization, and accuracy loss grows with sparsity.

Timeline (8)

RSS
  1. 2015

  2. 2018

  3. 2021

    Paper

    NVIDIA researchers published "Accelerating Sparse Deep Neural Networks", describing the 2:4 fine-grained structured sparsity pattern supported by Sparse Tensor Cores in Ampere GPUs and a prune-then-fine-tune recipe that preserves accuracy, so a regular sparsity pattern could be turned into hardware speedup.

  4. 2023

  5. 2024

  6. 2025

  7. 2026

See also (4)

Open full graph