Kirillov et al. at Meta AI released the Segment Anything Model (SAM) and the SA-1B dataset (1.1 billion masks on 11M images), turning object segmentation into a zero-shot promptable foundation capability driven by points, boxes, or coarse masks—quickly adopted for automatic inpainting masks in ComfyUI.
Segment Anything (SAM)
Also known as: SAM · SAM 2 · SAM 3 · Segment Anything Model
Plain English
Meta AI's family of visual segmentation foundation models that can isolate and track any object in an image or video from a single click, bounding box, or text concept prompt.
Technical Overview
SAM pairs a heavyweight MAE-pretrained ViT (or Hiera hierarchical ViT in SAM 2) image encoder with a lightweight prompt encoder and mask decoder for real-time interactive mask prediction. SAM 2 adds a streaming memory bank and memory attention module to track and segment objects across video frames and occlusions, while SAM 3 unifies open-vocabulary text concept segmentation with visual prompts.
Timeline (4)
2023
Paper2024
ReleaseRavi et al. at Meta AI open-sourced SAM 2 (Segment Anything in Images and Videos) under Apache-2.0 alongside the SA-V video dataset, introducing a streaming memory bank that attends over past frame predictions to track objects smoothly through video cuts and temporary occlusions.
- Release
Meta AI released SAM 2.1 checkpoints and full fine-tuning code in
facebookresearch/sam2, improving mask tracking on small objects, visually similar instances, and long occlusion intervals. 2025
ReleaseMeta AI launched SAM 3 with Promptable Concept Segmentation, enabling open-vocabulary text phrases alongside clicks and boxes to detect, segment, and track every matching object instance across images and video clips.