Mikolov et al. published Efficient Estimation of Word Representations in Vector Space (Word2Vec), introducing CBOW and Skip-gram to learn dense static word vectors where algebraic offsets capture semantic analogies.
Embeddings
Also known as: Vector Embeddings · Text Embeddings · Dense Representations
Plain English
Lists of numbers that capture the meaning of text, images, or code so similar concepts sit close together in mathematical space—powering semantic search, recommendations, and RAG retrieval.
Technical Overview
Embeddings project discrete inputs (tokens, sentences, code blocks, or image patches) into fixed-dimensional dense vectors via contrastive or bi-encoder training so cosine similarity reflects semantic relatedness. Modern embedding models use Siamese Transformers (Sentence-BERT), Matryoshka Representation Learning for truncatable dimensions, and CLIP-style cross-modal alignment for hybrid retrieval.
Timeline (4)
2013
- Paper
2019
- Paper
Reimers and Gurevych introduced Sentence-BERT (SBERT), fine-tuning Siamese Transformer encoders so sentence embeddings can be compared with fast cosine similarity—making semantic search and clustering practical at scale.
2022
- Paper
Muennighoff et al. released MTEB (Massive Text Embedding Benchmark), standardizing evaluation across retrieval, clustering, classification, and reranking tasks on Hugging Face for RAG and search models.
2024
See also (bkz.) (8)
Open full graphEmbeddings