Mathilde Caron et al. at Meta AI published Emerging Properties in Self-Supervised Vision Transformers (DINO), showing that self-Distillation & Synthetic Data with no labels causes ViT attention maps to spontaneously segment foreground object boundaries and semantic parts.
DINO (DINOv2 & DINOv3)
Also known as: DINOv2 · DINOv3 · Self-Supervised Vision Features
Plain English
Meta AI's family of self-supervised vision foundation models that learn rich spatial features—understanding object boundaries, 3D depth, and part correspondences—Purely from unlabeled images without any text captions.
Technical Overview
DINO trains a student Vision Transformer via self-Distillation & Synthetic Data to match the output distribution of an exponential-moving-average teacher network across multi-crop views. DINOv2 and DINOv3 combine image-level DINO loss with patch-level iBOT masked modeling, register tokens, and Gram anchoring on up to 1.69B curated images to produce frozen dense representations for monocular depth, segmentation, and VLMs.
Timeline (4)
2021
Paper2023
ReleaseOquab et al. at Meta AI open-sourced DINOv2 trained on 142M curated images (LVD-142M), delivering frozen universal visual features that matched or beat supervised models on monocular depth estimation, semantic segmentation, and image retrieval without fine-tuning.
- Paper
Darcet et al. published Vision Transformers Need Registers, revealing that large ViTs repurpose redundant background patch tokens as global memory registers and showing that appending explicit
[REG]tokens eliminates high-norm artifact spikes in dense feature maps. 2025
PaperMeta AI released DINOv3, scaling self-supervised vision foundation models to 7B parameters on 1.69B curated images (LVD-1689M) and introducing Gram anchoring to prevent dense patch-level feature degradation during long training schedules.
See also (6)
Open full graphDINO (DINOv2 & DINOv3)