David Rumelhart, Geoffrey Hinton, and Ronald Williams published Learning representations by back-propagating errors in Nature, demonstrating that backpropagation lets hidden layers in multi-layer neural networks learn useful internal representations instead of relying on fixed features.
ConceptLLMImageVideoCross-Modal
Backpropagation
Also known as: Backprop · Reverse-Mode Automatic Differentiation
Plain English
The core training algorithm for neural networks. It measures how wrong a network's prediction was and works backward layer by layer to calculate how every weight should be nudged to reduce future errors.
Technical Overview
Applies the chain rule in reverse mode from scalar loss to inputs, computing exact parameter gradients in a single backward pass whose cost is roughly proportional to the forward pass. Paired with gradient descent optimizers, backpropagation trains Deep Learning architectures end-to-end, with modern frameworks like PyTorch automating it through dynamic computation graphs.