IndexMapAbout
ConceptLLMImageVideoCross-Modal

Foundation Model

Also known as: Foundation Models

Plain English

A foundation model is trained on broad data so it can be reused for many tasks. For example, one language model can summarize documents, answer questions, and help write code.

Technical Overview

Pretraining produces a shared set of weights that transfers across tasks or domains. Users can specify a task in a prompt, or developers can adapt the model through post-training and methods such as lightweight adapters. Foundation models include language models, vision-language models, and generators such as Stable Diffusion. When many applications share a model, they can inherit the same limitations.

Chronological Entries (4)

#3Paper
AI-Assisted

Meta researchers introduced Segment Anything, a visual foundation model that produces a segmentation mask from prompts such as points or boxes. They evaluated whether the same model could transfer to new image distributions and segmentation tasks without task-specific training.

#2Paper
AI-Assisted

Researchers at Stanford’s Center for Research on Foundation Models used the term foundation model for models trained at scale on broad data and adaptable to many downstream tasks. They grouped examples such as BERT, GPT-3, and DALL-E under one term and noted that the limitations of a shared model can propagate to applications built on it.

#1Paper
AI-Assisted

OpenAI researchers described GPT-3, a pretrained Transformer that could perform tasks such as translation and question answering from instructions and examples in its prompt. They evaluated those tasks without updating the model’s weights for each task.

Direct Relations (1)