IndexMapAnti-DoomscrollAbout
ConceptLLMImageVideoCross-Modal

Backpropagation

Also known as: Backprop · Reverse-Mode Automatic Differentiation

Plain English

The core training algorithm for neural networks. It measures how wrong a network's prediction was and works backward layer by layer to calculate how every weight should be nudged to reduce future errors.

Technical Overview

Applies the chain rule in reverse mode from scalar loss to inputs, computing exact parameter gradients in a single backward pass whose cost is roughly proportional to the forward pass. Paired with gradient descent optimizers, backpropagation trains Deep Learning architectures end-to-end, with modern frameworks like PyTorch automating it through dynamic computation graphs.

Timeline (2)

RSS
  1. 1986

    Paper

    David Rumelhart, Geoffrey Hinton, and Ronald Williams published Learning representations by back-propagating errors in Nature, demonstrating that backpropagation lets hidden layers in multi-layer neural networks learn useful internal representations instead of relying on fixed features.

  2. 2019

See also (5)

Open full graph