Hello Guys,
Suppose that we have a data from a chemical process. There are some inputs and one output, the output depends on those inputs and also the time. I am gonna identify this process. So what is the difference between using simple MLP with Lagged output as an input (lagging the output as an input to the model, like 5 step before) and using recurrent neural network such as ELMAN or JORDAN without lagging the output, because i know JORDAN is just feeding back one step before as an input.
Thanks
https://pdfs.semanticscholar.org/7572/eb75115d351b20a3921e1fc325dee8bc01f3.pdf
http://www.cs.bham.ac.uk/~jxb/INC/l12.pdf
dear Pezhman,
it's a nice question. I just wanna share what i have tried the methods.
In the time series prediction, i use lagged output as inputs. That's quite good for accuracy. But sometime it's still too sensitive agains the change of input values. It's seemingly what we call overfitting. When i add the recurrent to the model, the outputs is better.
My Conclusion, The recurrent can avoid the overfitting of ordinary MLP.
nice to discuss with you :)
When you use a recurrent neural network you are doing more than a shift register of lagging inputs. When you do a recurrent neural network you usually train using back propagation through time so the weight of the lagging outputs (Jordan) or hidden layer(Elman) are trained by unfolding the network in time and average the weights through the propagation in time.
See: A guide to recurrent neural networks and backpropagation by Mikael Boden
RNNs are generative models in a fully probabilistic framework. They are connected to Particle Filters for HMMs (Hidden Markov Models). Follow:
- Choe et al., " Probabilistic Interpretations of Recurrent Neural Networks ", 2017 - https://www.cs.cmu.edu/~epxing/Class/10708-17/project-reports/project10.pdf
You may also check out my post related to one connex issue on ResearchGate: https://www.researchgate.net/post/What_are_other_modern_alternatives_to_Markov_Chain_models
Regards
And finally, as a big picture:
- M.A.R. Anjum, "A New Approach to Adaptive Signal Processing", 2015 - https://arxiv.org/ftp/arxiv/papers/1504/1504.06054.pdf
Regards
- Badges
- Science topic