Recurrent neural networks (RNNs) are a class of artificial neural networks for sequential data processing. Unlike feedforward neural networks, which process data in a single pass, RNNs process data … See more

Fully recurrent
Fully recurrent neural networks (FRNN) connect the outputs of all neurons to the inputs of all neurons. In other words, it is a fully connected network. … See more

Independently RNN (IndRNN)
The independently recurrent neural network (IndRNN) addresses the gradient vanishing and exploding problems in the traditional fully … See more

A continuous-time recurrent neural network (CTRNN) uses a system of ordinary differential equations to model the effects on a neuron … See more

Before modern
One origin of RNN was neuroscience. The word "recurrent" is used to describe loop-like structures in anatomy. In 1901, Cajal observed "recurrent … See more

An RNN-based model can be factored into two parts: configuration and architecture. Multiple RNN can be combined in a data flow, and the data flow itself is the configuration. Each RNN itself may have any architecture, including LSTM, GRU, etc. See more

Teacher forcing
An RNN can be trained into a conditionally generative model of sequences, aka autoregression.
Concretely, let us consider the problem of machine translation, that is, given a sequence See more