As we know the part of Feature Extraction is automatically done in Neural Network so is there any necessity of Feature Selection separately?
Dear Sayak,
In some problems, there might be redundant features existence of which may actually cause performance degradation. This is commonly known as "curse of dimensionality". Hence, application of dimensionality reduction techniques such as PCA (Principal Component Analysis) might provide performance improvement. An alternative solution could be application of feature selection techniques which iteratively remove redundant features.
I can suggest you look at the following course notes:
http://courses.cs.tamu.edu/rgutier/cs790_w02/l16.pdf
Best of luck,
Dear Paul,
Neural network performance can be degraded due to following issues,
1. Learning rate
2. Overfitting.
Specifically, overfitting is a problem in which the model is very good with training dataset, but not good at test dataset or in real application. Actually, overfitting can be occurred due to irrelevant or redundant features. In this case, feature selection or dimension reduction is a obvious solution. More about about feature selection or dimension reduction can be studied in the following references.
Chapter A Hybrid Feature Selection Scheme Based on Local Compactness...
Conference Paper Feature selection techniques for increasing reliability of f...
Conference Paper Filter and Wrapper-based Feature Selection Using Mutual Info...
Conference Paper Genetic Algorithm Based Discriminant Feature Selection for I...
Sincerely,
M M Manjurul Isilam
University of Ulsan, South Korea
Dear Islam,
I knew about the things that you mentioned. I am well aware of the techniques related to Feature Selection and Dimensionality Reduction. I am also able to distinguish between their similarities and dissimilarities.
But my question is different in all probability.
Dear AKAY,
Thank you for your inputs.
Neural networks definitely do not perform feature extraction automatically (unless you can afford to use deep learning because you have zillions of data). So feature selection is absolutely mandatory for neural networks as well as for any machine learning technique, unless you absolutely sure that all your candidate variables are relevant. Feature selection techniques are excellently covered in "An Introduction to Variable and Feature Selection", I. Guyon, A. Elisseeff, Journal of Machine Learning Research 3 (2003) 1157-1182.
CNN generate features using n-1 layers but the number if parameters is important. It is necessary to reduce features using evolutionnary algorithms . The chromosom is à set of bits" 1 the parameter is selected else is not considred. Concerning fitness function, : (1-alpha )*(number of selected features /length of chromosom) +alpha *rate recognition.
I hope that thé response us useful.
Thanks Nabil. Could you please share any article related to your answer. Would be of big help.
https://doi.org/10.1016/j.asoc.2017.11.006
This link for features selection
Thanks Gerard. I will have a look at the paper you mentioned. And yes, by mentioning Neural Networks I essentially mean Deep Learning.
Dear Sayak
You can use a pre-learned Deep network (e.g. Alexnet). You may/may not tune it with your data. Then the output before the final ANN (i.e. Softmax) will be the features you want.
Conference Paper Anuran call classification with deep learning
Hi Sayak, it is an interesting question indeed and like the NFL theorem, there is no right answer to this. My personal view is neural networks are inherently black box and when they would generate the non linear combinations the feature having less discriminating power will get lesser weights. I have worked little bit in feature selection and what I have understood is , it is compeletly dataset and domain dependent.
Very interesting question though , please keep me posted on your finding if possible.
Thanks Saptarsi. Could you share your work on Feature Selection? I am also working on a Feature Selection problem. Our goal is to devise an FS algorithm which will be as domain independent as possible.
Sure.
1) Unsupervised
Article A feature cluster taxonomy based feature selection technique
2) Supervised with a fusion technique
Article An approach of feature selection using graph-theoretic heuri...
3) Unified Method
Article A new hybrid feature selection approach using Feature Associ...
Some more
https://content.iospress.com/articles/journal-of-intelligent-and-fuzzy-systems/ifs2156
http://jucs.org/jucs_22_6/a_proposal_for_recommendation/jucs_22_06_0760_0781_goswami.pdf
Neural networks definitely do not perform feature extraction automatically.
You can use Convolution Neural Networks for automatic feature extraction and optimization.
Dear Padmavati,
I implicitly meant CNNs here. But how efficiently will CNNs be in case of datasets other than images? If so, could you please tag me to some of the articles related?
Feature selection can automatically be performed by neural networks in conjunction with L1-regularization. This can easily be implemented using the weight decay technique with the corresponding protocol. In this case, the feature selection is a part of network pruning and contrasting procedures, which were very popular 20-25 years ago at the age of shallow neural networks. Automatic feature selection using the above-mentioned approaches was one of the most attractive features of out 20-years old software NASAWIN for conducting structure-activity studies with neural networks.
- Badges
- Science topic