Using swarm for feature selection
Firts the data set is passed to the algo and thealgo generate some solutions each soltion is a candidate subset. At each iteration the fitness of solution is calculated using a classifier accuracy and the best solution is returned and so on for a number of iterations .At the end the optimal subset is returned
1- how I can choose the algo parameters?
2-what can I do with the selected subset ?
3-how many runs of algo?
Thank you
Hi Ahmed,
You can use swarm algorithms for parameter optimization for some prediction models. For example, you can use swarm algorithms to optimize the parameters of coefficients of logistic regression to classify the output. If the coefficients close to zero, you can delete related features.
You can look below paper...
https://www.researchgate.net/publication/303692790_A_Meta-heuristic_LASSO_Model_for_Diabetic_Readmission_Prediction
Conference Paper A Meta-heuristic LASSO Model for Diabetic Readmission Prediction
salem,
1- The algorithm parameters tuning still an open problem. Some researchers tuned them experimentally (we tuned one parameter at a time while we fixed the remaining ones ). Heuristic could be used to tune the parameters.
Also, in the paper above, authors have mentioned that all the parameters of their algorithms are chosen by trial and error (i haven't idea about this mehtod ).
(The paper is :http://content.iospress.com/articles/journal-of-intelligent-and-fuzzy-systems/ifs1867)
2- Only features included in the best selected subset are used in the training and test data (train and test data contains only the selected features) .
using only these features, the train data are used to train a classifier (generally the same used for the fitness) then a test data are classified and a classification measures are observed to evaluate the feature subset quality.
3- usually, the FS algorithm is executed 30 times ( the average values are returned ).
Best regards
Hi Ahmed,
Please find the answers below.
1- how I can choose the algo parameters?
Have a look at this article:
Moraglio, A., Di Chio, C., Poli, R.: Geometric Particle Swarm Optimisation. In: Proceedings of the 10th European Conference on Genetic Programming, Berlin, Heidelberg, 125-136, 2007.
2-what can I do with the selected subset ?
The selected subset should be passed to another method (an evaluator), then this subset will contribute in building the prediction model.
3-how many runs of algo?
You can select random seed no. = 1 for 20 iterations.
HTH.
Samer
hello when you use classification accuracy it is called a wrapper model and as for each iteration you need to build the model, this is not really great efficiency wise, yes performance may be better. You can use a fitness function like CFS ( Correlation based feature selection) or mRMR ( minimum redundancy maximum relevance) . Feature selection i.e. finding an optimal subset is a NP-Complete problem hence several meta-heuristic strategies like GA, PSO etc. can be implemented. Now coming to your questions
1) Trial and Error and literature study
2) Those are the features you should use in your final model, however you must get it vetted by a domain expert.
3) Same as 1, there are several convergence criterias that can be looked at.
HTH
Read my chaotic chicken for feature selection , Follow algorithm step , data set , how to setup experiment and think how to present a novel and resolve research gap with swarm in feature selection
Thank you Dr/Khaled Ahmed
please i need the full-text of this paper
- Badges
- Science topic
- Similar topics
- Computer Science and Engineering
- Algorithms