SVM - RFE is a binary classifier and removes only one feature in each iteration. I want to eliminate a set of features. in many papers they have eliminated the lowerst ranked 50% of features, in some they remove 1/4th of feature. Is there any method to compute this threashold from weights of each feature.
I have an idea, of sorting the weights of each feature and to find the longest difference between the weights of the sorted features. then removing all the features below the longest gap.
Hi, the Feature selection extension in RapidMiner has an operator called "Recursive Feature Elimination" which includes a parameter "ratio" that controls the percentage of features to keep (and eliminate) in each iteration. Attached is a RapidMiner code that eliminates 50% of features in SVM-RFE (copy-paste the code to xml tab, then press green v-sign and then run the process). Note that you may optimize the %-elimination parameter.
Cheers
P
There is no universal rule as the proportion of features to eliminate could be problem/data dependent. In other words, this is a hyper-parameter you have to fix and indeed you are not restricted to eliminate a single feature at a time.
Dropping 50 % of the features (or the alternative you suggest) could be somewhat contradictory to the objective of gradually removing features. As a rule of thumb, I suggest 10% or 20 % at each round. The benefit of defining this threshold as a fixed fraction of the number of (remaining) features is that, in absolute terms, you eliminate features in larger chunks at the beginning of the process and you gradually converge to single feature elimination when few features are left.
Multiple features (any number of features) can be eliminated at a time. spider toolbox has a implemented the same in MATLAB, called SVM-RFE.
- Badges
- Science topic
- Similar topics
- Mathematics
- Algebra
- Matrix