History
Who invented the LSTM architecture in deep learning?
Sepp Hochreiter and Jürgen Schmidhuber in 1997.
Card 1 of 276. Answer: Sepp Hochreiter and Jürgen Schmidhuber in 1997.
Sepp Hochreiter and Jürgen Schmidhuber in 1997.
Card 1 of 276. Answer: Sepp Hochreiter and Jürgen Schmidhuber in 1997.
About · Quizzes · History of AI · To suggest new questions or report an error send me a dm.
Sepp Hochreiter and Jürgen Schmidhuber. The standard reference is their 1997 paper "Long Short-Term Memory" in Neural Computation (volume 9, pages 1735 to 1780), which followed a 1995 technical report at TU Munich. An LSTM is a recurrent network unit with a memory cell whose contents can survive across long stretches of a sequence.
The idea grew out of Hochreiter's 1991 diploma thesis at TU Munich, written in German and supervised by Schmidhuber, which analyzed why error gradients in recurrent networks vanish or explode over many time steps. The fix was the constant error carousel: a cell connected to itself with weight 1, so error flows back unchanged, guarded by multiplicative input and output gates. The 1997 paper bridged time lags of more than 1,000 steps.
The 1997 version had no forget gate; Felix Gers, Schmidhuber and Fred Cummins added it in 1999. LSTMs later powered Google's 2016 neural machine translation system before Transformers took over most language tasks, and in 2024 Hochreiter's group revisited the design as xLSTM.