Recurrent Neural Networks have cyclic connections. I was wondering how the termination case is encoded in such networks.
Thank you
Patricia
There are several options.
Many of them are based on network convergence to the steady state.
When the signal processing in real-time useful alternatives. In particular, one of them considered in my articles on recurrent neural networks with controlled synapses.Take into account the priority of short connections between neurons and spatial shifts of signals.
Dear Patricia,
The nature of recurrent connections is such that temporal dynamics is incorporated in the system. In other words, while feedforward networks can be seen as atemporal signal processing systems, recurrent neural networks need to have a notion of time that enable output signals to be fed back as input to the same neurons.
As such, RNNs do not, generally speaking, terminate their computation after a fixed number of steps. In some cases, when for example oscillatory dynamics emerge, the state of the network changes indefinitely over time, and a termination criterion is not contemplated. In other cases, the network might converge to a stable state, and remain ins such a state unless the inputs are disturbed. In this second case it is possible to monitor the variations in the state of neurons, and decide that the computation has reached a stable state when the smallest oscillation in the network (from one step to the next step) is less than a fixed small constant.
- Badges
- Science topic