Geoffrey Hinton, Simon Osindero, and Yee-Whye Teh published A fast learning algorithm for deep belief nets, showing that greedy layer-wise unsupervised pretraining could initialize deep multi-layer networks—reigniting research interest under the deep learning banner.
Deep Learning
Also known as: Deep Neural Networks · DNN
Plain English
A branch of machine learning that trains multi-layer artificial neural networks to learn patterns directly from raw data—such as text, images, or audio—instead of relying on hand-crafted rules or features.
Technical Overview
Stacks parameterized linear and nonlinear layers trained end-to-end with Backpropagation and stochastic gradient descent, using GPU acceleration and autograd frameworks to scale to billions of parameters. Modern deep learning evolved from CNNs and RNNs to Transformers and foundation models.
Timeline (3)
2006
Paper2012
MilestoneAlex Krizhevsky, Ilya Sutskever, and Geoffrey Hinton presented AlexNet at NeurIPS 2012, training a deep convolutional neural network on NVIDIA GPUs to win ImageNet by over 10 percentage points and triggering widespread adoption of GPU-accelerated deep learning.
2024
MilestoneThe Royal Swedish Academy of Sciences awarded the 2024 Nobel Prize in Physics to John Hopfield and Geoffrey Hinton for foundational discoveries enabling machine learning with artificial neural networks, recognizing Hopfield associative memories and Hinton's Boltzmann machines.
See also (7)
Open full graphDeep Learning