IndexMapAnti-DoomscrollAbout
See also
ArchitectureLLMCross-Modal

Recurrent Neural Networks

Also known as: RNN · LSTM · Long Short-Term Memory · Seq2Seq

Plain English

Sequence models that read text or time-series data one token at a time, passing an internal memory vector forward from each step to the next like a running summary.

Technical Overview

Maintain a recurrent hidden state h_t = f(h_{t-1}, x_t) updated sequentially across timesteps. Gated variants like LSTM mitigated vanishing gradients over longer sequences, and encoder-decoder Seq2Seq architectures introduced Attention before the Transformer replaced sequential recurrence with parallelizable self-attention.

Timeline (3)

RSS
  1. 1997

    Paper

    Sepp Hochreiter and Jürgen Schmidhuber published Long Short-Term Memory (LSTM), adding input, forget, and output gates around a constant error carousel so recurrent networks could retain information across hundreds of steps without vanishing gradients.

  2. 2014

    Paper

    Ilya Sutskever, Oriol Vinyals, and Quoc Le introduced Sequence-to-Sequence (Seq2Seq) learning with multi-layer LSTMs, showing that an encoder-decoder neural network could map variable-length input sentences directly to variable-length translations.

  3. 2017

See also (4)

Open full graph