Hi, now I want to introduce the Convolutional Neural Network to variable selection. I have a high dimensional gene data and some labels, and I want to find some important variables. However, since the data is sparse, the CNNs is not that powerful as it is used in image classification. Now, I want to change the traditional pooling method in CNNs and make variable selection method,such as lasso, as pooling strategy instead. In this way, I think the dimensional can be reduced efficiently with some Variable selection method such as (Lasso,LAR,SCAD). Do you think it will work? Welcome to discuss with me. Thank you very much!
Hi Chen,
In fact, there are a lot of techniques that can be used to reduce the dataset features (i.e., dimensions) such as applying PCA or using any other feature selection techniques. In my point of view, you can follow entirely different method to ensure having highest classification accuracy and at the same time reduced features. You can randomly select different combination of features to generate many subsets, then apply the CNNs on all the subsets and compare the results. Finally pick a subset that gives you the highest accuracy.
Kind regards
Ahmed
Interesting question!
The first thing to say is that PCA/SVD can indeed be used in the inner layers of the network, as Ahmed mentioned. They can achieve (at least to some degree) some of the features that you might expect to get from lasso, but they fall short too I think.
I guess what you'd really like is to obtain is a lasso path for your problem. That would be very useful! An immediate problem that strikes me is that CNNs optimise over a non-convex hypothesis space. i don't know if this would necessarily prohibit the calculation of lasso for the CNN (must think more on that), but possibly the best you would get is a particular lasso path, rather than THE lasso path which is what we are used to.
Niall
I used bio-inspired optimization algorithms like PSO and BAT for features reduction (for OCR applications), they achieve good results and superior time consumption.
Last October, I solved your problem completely by the Matroska feature selection method for microarray data. See and download my papers from RG.
- Badges
- Science topic