When using RNN, we often have the problem of vanishing gradient or exploding Gradient. Is there any suggestions or proposition to resolve such as problems?
Thank you.
kind regards
Hi,
Gradients Vanishing is very small close to zero
Exploding Gradients is very big
This occurs in the training phase
Dear El Adoui and your lab and others on this string,
We have shown that in Genetic Neural Network spacextime Differentiable Strings in the space of models and genetic data matched in their quantum mechanical fields by genetic algorithms which are diffeotopically embedded with Gauss-Markov quantum group homology , vanishing and exploding gradients can be embedded into programmes with the usual FTP and can be solved by randomised genetic algorithms over the internet. We have carried out by a number of spacextime relativistic Higgs-Mallick (Mallick, Hamburger & Mallick (2016, 2017, 2018)) mechanics experiments using particle flow group homology and found to be solvable by our established Systems Classification and Systems Integration Theorems (Mallick (1993, 2012, 2015)). This shows that all such experiments are Fundamentally in terms of the standard model of Physics using "sustainable"particle flows of phononic-photonic duality using string matching field theory (Mallick, Raychaudhury, Mallick & others (2018)) in AGGNNNetwork String duality Theory. Hence, without meaning to provide convoluted solutions the Econophysical cybernetic explanation to your vanishing and exploding gradient problems are observable in our above papers and an empirical analysis can also be found in my research monograph based on my PhD thesis at NYU in Mallick (2009).
S.K.Mallick (CA)
for S.K.Mallick, S.Raychaudhury, S.Mallick and others ,
The RHMHM School,
USA, Japan, Russia, India
- Badges
- Science topic