Dear Experts,
I trained the neural network with six inputs using the backpropagation algorithm. The target is 0 and 1 which is needed to be classified. The performance of NN model is 98.4% for all correct classification from the confusion matrix. Afterward, I fed the new unseen data to produce the predicted values and compare with the actual output. I noticed that the model works poorly with the unsatisfied results. Therefore, I would like to ask you for a suggestion, comment, solution and so on.
Thank you in advance for your help.
Thammavongsy
How did you split your data in training, test and validation sets?
How big is your data set? How big is your network?
In addition to the questions asked above, are your classes equally distributed, ie. do you have 50% 1s nd 50% 0s, or some other distribution?
When training your network, do you evaluate the validation accuracy at each epoch?
It seems like you're overfitting your network to the training data. You may need to shrink your network a bit, employ some regularizarion techniques, or add Dropout layers.
Try predicting the opposite result if you have a binary classification. Try entering zeros as well as ones in the selection. It is necessary to compare with the probability of random guessing
First, thank you for your comments, Roman Liessner . I did it with 80% of the training set and 20% of the testing set. The entire data sample is 8744 samples (100%). I utilized 3 hidden layers contain 45neurons. I can notice that the sample of class 0 is more than class 1. Do you have any comments or ideas to help me, please.
Regards,
Phimmasone
Yes, your comments are useful, Patrick Timmons . Class 1 and Class 0 are not 50%. I found that the dataset of two classes is imbalanced. Once again, how is this issue improved or etc? Thank you for your comments.
Regards,
Phimmasone
You can specify a class weight to adjust for the imbalanced number of classes. See this thread for an explanation for doing this in Keras, other frameworks would be similar I expect.
https://datascience.stackexchange.com/questions/13490/how-to-set-class-weights-for-imbalanced-classes-in-keras
How is your distribution between 1 and 0?
Here some techniques to handle imbalanced data:
https://www.kdnuggets.com/2017/06/7-techniques-handle-imbalanced-data.html
https://towardsdatascience.com/methods-for-dealing-with-imbalanced-data-5b761be45a18
In addition, it would make sense to divide your data set into three parts instead of two. For example 60/20/20
For training you use the 60 % and the 20 % for early stopping.
And only if your model still works well on the 20% of the test set you should use it.
- Badges
- Science topic
- Similar topics
- Organizational Psychology
- Training