I am working on gait analysis using a feed-forward NN. When I use data from the accelerometer (3 columns for each axis if stored in a data frame), I get an accuracy of 95%, and when I used data from the gyroscope (3 columns for each axis if stored in a data frame), I get an accuracy of 85%.
Now, when I try to use data from both of these above mentioned sensors (6 columns if stored in a data frame), the accuracy drops to 66%. Being quite new to this, I am not sure why is this happening? Shouldn't accuracy be in the same range as before?
What can be the possible reason for this?
Thanks in advance!
Assuming both datasets represent the same problem domain, I suspect your model was overfitting when you were training it on the separate datasets mainly due to insufficient data. The scored accuracy when combining both data, I believe, is the best model not being affected by overfitting considering the general rule more data results in a more robust model. If you feel 66% is very, you can follow some tricks most of which are discussed in the following book, I highly recommend.
https://www.amazon.com/Deep-Learning-PyTorch-practical-approach/dp/1788624335
Thanks Ephrem Admasu Yekun for your insight and recommendation.
But shouldn't training accuracy be 100 if the model is overfitting and testing accuracy smaller than training accuracy? The accuracies I mentioned in the problem are testing accuracy and training accuracies were also in the same range.
Am I missing any major concept here?
In that case, it's hard to identify the problem but did you try different types of NN architectures to build model for the separate, and combined datasets? Did you apply some tricks (dropout, regularization, feature engineering etc) as mentioned in the book I recommended? How confident are you those two separate datasets represent the same domain? Or are there reputable sources suggesting the datasets be used in combination?
Ones I answer these questions and if the problem is not fixed, then I would chek my code for some bugs.
Did you look at the data? The situation that you describe suggests that each sensor delivers self-consistent information, but that the information of the sensors are mutually contradictory. For instance, if the x-axis of one sensor is oriented, say, vertically upward, and the x-axis of the other sensor is oriented vertically downward, the combination of the two sensor data might be intractable by any machine learning method.
Gérard Dreyfus thanks. I looked into the data and it seems that what you have pointed out is partially correct in this case. Thanks a lot!
Glad to have been of assistance.
As you are a newcomer to the field, always keep in mind that machine learning is data-driven, hence data quality should be the first concern.
Also, by all means do not try to learn machine learning from software users' manuals, such as the above-mentioned Pytorch, and many others. Machine learning is science, but it tends to become a bag of tricks. Read scientific textbooks and articles written by scientists. Keep away for tricksters who will almost invariably lead you on a wrong path.
- Badges
- Science topic