I have a data set with 91 Obervsation with 700 features. I have reduced the dimension of data using PCA? then I split the data set into 70/30. After that applied kfold =10 cross-validations on training data set and accuracy over each fold is varied a lot.
this is accuracy over each fold?
fold1 = 0.71
fold2 = 0.83
fold3 = 0.50
fold4 0.29,
fold5 = 0.67 ,
fold6 = 0.33
fold7 = 0.14
fold8 = 1.00
fold9 = 0.33
fold10 = 0.14
is it correct to have such variation? if not then how can I solve that problem?
how can I improve my accuracy over each fold at least it should not vary much?
what will be the possible reason for that? why it vary a lot?
I am looking for an answer to these questions
I think your method is Okay, however., there may be some problem with your equations or code. Also you may change the size of training and testing data for validation. Also during PCA you also need to have appropriate sizes of the reduced dimensionalities.
I assume you are using a computer package to do your 10 fold cross validation. Thus it should split your data rather than you. See Steyerberg Clinical Prediction Models for full details. Best D Booth
Possibly there are large variations among the sample features. One solution is to use a better feature set that provides small intra-class differences and large inter-class differences.
David Eugene Booth, I have tried both ways to do 10 fold cross-validation using caret package as well as manually implement kfold function but the result seems not to be improved.
- Badges
- Science topic
- Similar topics
- Correction