While using kfold cross validation on my deep neural network recommender system model, for different random_states of train_test_split I am getting huge variation in r2score for test data(0.2 to 0.9) while my training r2 score is always more than 0.93
Thanks for the reply,
I just found the variation in r2_Score is due to validation result of very first fold is very poor(say 0.2) and then for the other folds validation improves to more than 0.9 . How to improve r2 score for first fold? (initializers didnot help)
I have data of 4700 samples. Is it Ok if I use 20 folds? althougj my test set will be just 5% every time but my average r2 score improvesy
Generally, if you have very high training scores and poor validation scores it means you are overfitting on the training data. You can do some changes to your model (like hyperparameters optimisation) and try to see the learning curves or deviance curves to verify if you are overfitting and choose the best model with optimised hyperparameters. In case of neural networks you can also introduce early stopping conditions to your model.
- Badges
- Science topic
- Similar topics
- Organizational Psychology
- Training