i am working in data classification using metaheuristic ( and swarm )feature selection , i appply svm,knn in the original dataset without fs (100 features) and record the accuracy in weka tool
and then make a wrapper feature selection ( swarm + knn as a fitness function ) and produce a subset of features ( f.e 20) and take only20 features and make a classification again in weka
compare the without/with swarm/2 filter based fs in weka
1- the acuracy here is the best solution fitness in swarm algorithm or the accuracy of classification after fs ?
2- how many datasets and their sizes ( i use small 20 features , this is true)
3- why to make 20 runs of algorithm and is it neccessary ?
thank you
Hello,
I'm not sure if I understood correctly.
You are working with a certain dataset X which consists of 100 features,
the goal is to select the best combination Classifier-Feature subset?
In such a case, if you are splitting the dataset X into Training, Validation and Testing, you should repeat the experiment many times, since you can obtain different results according to the split (which is random).
Performance should be averaged over many trials.
If your goal is to compare different feature selection techniques, they should be applied on the same split of the dataset, so as to be fair. Then, in line with previous point, repeat for many trials (the larger the better).
- Badges
- Science topic
- Similar topics
- Computer Science and Engineering
- Artificial Intelligence