Hi all,
One of the most common techniques to solve the vanishing gradient problem in deep neural networks is using different skip connection (shortcut) schemes that enable the gradient to be directly backpropagated to earlier layers of a deep network. Do you think it is worthwhile to investigate the possibility of employing skip connection modules in shallow neural networks?
Thanks so much.
I think you can traverse this area. Similar studies are also available. Here you can find one
https://link.springer.com/article/10.1007/s11390-019-1950-8
Dear Abouzar Choubineh
These articles might be helpful to connect all the dots together, have a look:
1. https://openreview.net/forum?id=BJlRs34Fvr
2. https://www.researchgate.net/publication/339301014_Skip_Connections_Matter_On_the_Transferability_of_Adversarial_Examples_Generated_with_ResNets
3. Conference Paper The Importance of Skip Connections in Biomedical Image Segmentation
- Badges
- Science topic
- Similar topics
- Biological Science
- Bioecology
- Systems Ecology
- Landscape Ecology
- Connectivity