OpenAI published Improving Language Understanding by Generative Pre-Training, introducing GPT-1 (117M parameters) and demonstrating that unsupervised next-token pre-training on a decoder-only Transformer followed by supervised fine-tuning transfers strongly across NLP benchmarks.
GPT
Also known as: Generative Pre-trained Transformer · GPT-1 · GPT-2 · GPT-3 · GPT-4 · GPT-4o · GPT-5
Plain English
OpenAI's flagship family of decoder-only language and multimodal models that pioneered generative pre-training, few-shot in-context learning, and mainstream conversational AI.
Technical Overview
Built on autoregressive decoder-only Transformer stacks trained with next-token prediction and refined via RLHF and test-time reasoning, the GPT lineage scaled from the 117M-parameter GPT-1 through 175B-parameter GPT-3, multimodal GPT-4/4o, and unified reasoning routers in GPT-5 and GPT-6.2.
Timeline (7)
2018
- Paper
2019
- Release
OpenAI unveiled GPT-2 (1.5B parameters), trained on WebText, showing that scaling decoder-only language models unlocks zero-shot task transfer across summarization, translation, and question answering without task-specific fine-tuning.
2020
- Paper
Brown et al. at OpenAI published Language Models are Few-Shot Learners, scaling GPT-3 to 175B parameters and establishing in-context learning—where prompting with a few examples replaces gradient updates—as a defining paradigm of foundation models.
2023
- Release
OpenAI released GPT-4, adding vision inputs alongside text and posting major gains on coding, bar-exam, and reasoning benchmarks, setting the frontier capability bar for commercial foundation models.
2024
- Release
OpenAI launched GPT-4o ("omni"), an end-to-end multimodal model processing text, audio, and vision within a single neural network for low-latency real-time voice and visual interaction.
2025
2026
See also (bkz.) (13)
Open full graphGPT