What does GRU mean?

GRU stands for Gated Recurrent Unit, a recurrent neural network cell that uses gates to decide how much of its past state to keep at each time step. Kyunghyun Cho and colleagues proposed it in a 2014 EMNLP paper on encoder-decoder models for machine translation, as a simpler alternative to the LSTM.

It has two gates. The reset gate rₜ controls how much of the previous state feeds the candidate state, h̃ₜ = tanh(W xₜ + U (rₜ ⊙ hₜ₋₁)), and the update gate zₜ blends old and new: hₜ = (1 − zₜ) ⊙ hₜ₋₁ + zₜ ⊙ h̃ₜ. There is no separate cell state, and with three weight blocks instead of four, a GRU has about 25% fewer parameters than an LSTM of the same size.

The usual follow-up is GRU versus LSTM. A 2014 comparison by Chung, Gulcehre, Cho and Bengio found the two comparable, and neither has won outright since, so in practice you try both. Both ease the vanishing gradient problem of plain RNNs.