My research involves speech/chatter discrimination. I have a data set of audio files comprising 2 classes (speech, chatter). I have 300 audio files belonging to the speech class and another 300 for the chatter class (600 audio files in total). I know that I need to perform a 10-fold cross validation to evaluate the classifier. I just wanted some confirmation about the process I am going to follow, which I will describe now.
1. Divide the dataset into 10 subsets (60 audio files per subset - 30 speech, 30 chatter)
3. Train on 9 of subsets (540 files) and then test on the 1 remaining (60 files)... and repeat for all 10 subsets
Is this correct?
Yes the procedure you describe is correct: 10 folds, use each fold for testing and the rest of the data for training. Repeat for each fold and compute the average performance measures (f1, accuracy etc).
However, for the particular case of audio classification, fold definition should not be done using a random permutations. Instead one should define each fold in a way to include samples from the same recording or with similar characteristics in the same fold. In your case, for example, samples from the same speaker should be in the same fold, otherwise, you could end up in bias, since samples from the same speaker would be both in the training and testing datasets at the same time (which is kind of "cheating"...).
Theodoros Giannakopoulos i just saw your comment and it actually discusses the exact problem facing now with my Baby Cry dataset, i split my 5 sec audios into 2 sec segments so my training set has segments from the same speaker and i am getting high accuracy from the bias happening , so how can i keep samples from the same speaker be in the same fold, would you please explain ?