Suppose i am building a machine learning model, the data consists of 100 features (say), After feature selection i concluded that only 10 features are required for the classifier. So for a new classification (with unknown label/class) do i need to calculate 100 features or only selected 10 'best' features ?
Dear Joseph
This is an important question.
If you let the all features to be included in your final model, the answer to your question is yes.
However, I usually use importance value to select a limited subset of the features that are important most.
Regards,
Hormoz
I would calculate the 10 features that are required. However, it is important to be sure that the remaining 90 features are not relevant. A good way to do this is to do a lot of exploratory data analysis and visualizations during your data preparation stage. I hope that helps
It depends on the discrimination ability of the number of features in your algorithm. You to investigate this issue as part of your work.
Suppose a company needs to hire ten engineers to develop a new project. It advertises the job proposal, gets 100 applications, submit all applicants to its usual selection procedure, and selects ten of them. Would the company hire all 100 applicants, or hire the ten selected applicants only? That is basically the question you are asking.
The purpose of feature selection is feature selection: select the features that are most relevant to the task (classification, regression, ...) that your learning machine is supposed to perform. There exist some good, statistically meaningful feature selection methods. For an excellent introduction to feature selection, read the book "Feature extraction: foundations and applications" by I. Guyon et al (Springer).
After performing feature selection, the features that are not selected are discarded.
If Your new 10 features selected are good enough to overcome different aspects of classifications for your new classification then your answer is yes but it is actually depends on your new classification that what you wanna classify is something related to your previous classification which you extracted features regarding to that or not.
Question: "After feature selection i concluded that only 10 features are required for the classifier."
Answer: It Depends, which feature selection method you are applying, are they directly interpreting the features (for example NMF, encoder models) or transforming to other space via PCA, KPCA, etc. However, the best way is to observe the class-dependent individual 10 features distribution, then you can build the joint distribution model (probabilistic) or fit other ML models for your final classification.
Question: "So for a new classification (with unknown label/class) do i need to calculate 100 features or only selected 10 'best' features ?"
Answer: It again depends on the technique of the best feature selection. If you directly can choose the 10 best features from the data itself then go for 10 features directly, otherwise, you have to compute all 100 features then map to 10 dimensions by the transformation rule.
The feature selection process is mainly performed to reduce the number of features while maintaining the good performance of the classifier (or even improving it by discarding features that only add noise to the system). It has to be assumed that the training dataset is rich enough to rely on the best-selected features, which will then be used further on. It does not make sense to recalculate all features if the results were good with the training dataset.
- Badges
- Science topic
- Similar topics
- Data Mining
- Classification