I have developed a deep learning neural network which analyzes the heating pattern of alternators. I am feeding around 3900 data points as input and an equal no. as target sets. The network is able to retrace the non linear curve (as indicated in the picture), but at the end of each learning iteration, it only generalizes for the last 100-200 points . The same case happens when I test it. It gives accurate values only for the last 100-200 data points. How do I improve on this?
HAve you tried adding drop out? What are your regularization techniques?
Lucas Soares I have not added any regularization in my code. And what is a drop out?
Drop out is when you add a small probability of a certain node to be inactive so that in training your network does not rely so much on that node. If you do someting like this the network becomes better at generalizing (usually) because it learns other pathways to the correct output. But you can also try batch normalization and some other techniques...and see what works best
If you have some literature on it, could you share it with me?
Sure, i think these two links cover well:
https://www.datascience.us/neural-net-dropout-dealing-overfitting/
https://medium.com/@amarbudhiraja/https-medium-com-amarbudhiraja-learning-less-to-learn-better-dropout-in-deep-machine-learning-74334da4bfc5
One of the best source i have seen to go from basics to complex is Andrew's Ng deep learning course on coursera. Very well explained and covers the topics well.
THe youtube video of that course about dropout is this one:
https://www.youtube.com/watch?v=D8PJAL-MZv8&list=PLkDaE6sCZn6Hn0vK8co82zjQtt3T2Nkqc&index=6
Another awesome source to understand regularization.
Cheers!
Have you consider using some temporal information? Your data seems to me to be a time-series, using RNN (LSTM's) could be a good idea. Cheers.
you can use dropout in fully connected layers, batch normalization after convolutional layer can make your network more general.
- Badges
- Science topic