I heard of stepwise feature selection methods for regression problems. How can we perform stepwise feature selection methods when dealing with neural networks? Isn't it computationally expensive? Is there any other wrapper method that can perform well for neural networks learning procedure?
@Satya,
If you have NeuroSolutions software you can use "Sensitivity Analysis" feature of the software and rank your features and then do selection in a step-wise way manually.
If you do coding using Matlab or similar platforms, you can use some optimization technique like "Genetic Algorithm" for selecting best features automatically.
Dear Satya,
Yes this is computationally expensive because for each feature subset generated in stepwise feature selection algorithm you train the neural network to evaluate this subset.
But there is another one problem here: this evaluation is unreliable because the network does not give stable results. So it is hart to detect globally optimal feature subset.
Thank you Grzegorz Dudek, is there any alternative and optimal method to do feature selection in neural networks?
You can get sometimes better results using global optimization methods such as genetic/swarm algorithms or tournament searching (see below). But due to unstable results of NN learning (NN are learned using gradient methods leading to local minima) no methods give you globally optimal feature subset.
Conference Paper Tournament Searching Method to Feature Selection Problem
In my opinion the best way to achieve a feature selection using NN is to evaluate each subset of inputs with a K-fold cross-validation repeated N times. Otherwise you can try to assess the information content of the inputs using some statistical analysis or mutual information (see Battiti, 1994)
http://sci2s.ugr.es/keel/pdf/algorithm/articulo/battiti.pdf
I think ANN can find the supper complicated correlations among variables. Even if you add some irrelevant variables it can handle it to some extend. So there is not much point in feature selection using step-wise methods. What do you think?
It can depend on the problem at hand as well. e.g. you can remove a few features which you think do not make sense in the problem you're trying to answer.
Similarly, you can also remove linearly dependent features using a pearson correlation test.
- Badges
- Science topic