IndexMapAbout
ConceptLLMImageCross-Modal

Embeddings

Also known as: Vector Embeddings · Text Embeddings · Dense Representations

Plain English

Lists of numbers that capture the meaning of text, images, or code so similar concepts sit close together in mathematical space—powering semantic search, recommendations, and RAG retrieval.

Technical Overview

Embeddings project discrete inputs (tokens, sentences, code blocks, or image patches) into fixed-dimensional dense vectors via contrastive or bi-encoder training so cosine similarity reflects semantic relatedness. Modern embedding models use Siamese Transformers (Sentence-BERT), Matryoshka Representation Learning for truncatable dimensions, and CLIP-style cross-modal alignment for hybrid retrieval.

Timeline (4)

RSS

2013

2019

2022

2024

  1. Release

    OpenAI launched text-embedding-3-small and text-embedding-3-large trained with Matryoshka Representation Learning, allowing developers to truncate embedding vectors to smaller dimensions at query time to trade off vector-store memory and retrieval accuracy.

See also (bkz.) (8)

Open full graph
3 more connections