I think there is no direct answer to that question, furthermore there are multiple features selection algorithms which can provide different outcomes. The basic difference is that PCA transforms features but feature selection selects features without transforming them. PCA is a dimensionality reduction method but not feature selection method.

PCA drawbacks:

- The new principal components are not interpretable.

- You have to tune a threshold for cumulative explained variance.

PCA advanges:

- Perfoms well

- Versatile method

Feature selection can be achieved using correlation techniques/ information gain/ training a model on different subsets of features .

Please refer this link for more details:

https://machinelearningmastery.com/perform-feature-selection-machine-learning-data-weka/

Ise Wrapper evaluator,ifyou are usig Weka or Rapid miner

Moreover,i used Forward and backward su set elimination, optimization or recursive elimination functions. They all are good for feature selection.

Greed algorithm and rankers are also better.

Thank you Dr. Iván Contreras , Dr. Tharmakulasingam Sirojan , Dr. Sachin Kumar for your answers.

I have used fiter-based feature selection to determine the best feature subset. Moreover, the contribution of each feature to predict the target value can be calculated and identified based on Information Gain. Hence, the result of feature selection is more meaningful than Principal Components (PCs) in PCA as explained by Dr. Iván Contreras Is my opinion is make sense? I might need a broader view on this issue.

One big advantage of using PCA is, that the resulting principle components (PCs) are uncorrelated. This makes subsequent modeling a lot 'cleaner'.

I partly agree with Iván's comment that the PCs may not be as interpretable as the original variables. But in my experience, if you have a close ook at which variables have high factor loadings (correlation with a specific PC), in absolute sense, then you are often able to attach meaning to that PC.

If you don't want to use the PCs themselves in subsequent modeling (a drawback being that you need all underlying variables to calculate the value of a PC), you can also choose one high-loading variable for each PC. These variables will not be strongly correlated either.

Variable selection methods often give different results depending on the order in which variables are included in the analysis, which makes sense btw: the effect of variable A on target variable Y may be quite different depending on whether another variable B is already in the analysis.

The above doesn not mean I never use variable selection techniques, just that PCA has often proved to be valuable to me.

You may want to read Chapter 6: Linear Model Selection and Regularization and section on Principal Components Regression from James, G., Witten, D., Hastie, T., & Tibshirani, R. (2013). An introduction to statistical learning: with applications in R. Springer texts in statistics (Vol. XIV). http://doi.org/10.1007/978-1-4614-7138-7

In fact, PCA and feature selection complement each other. They are not at all in opposition.

When handling high dimensional data, succeeding in identifying first a lower dimensional subspace is of primary interest subsequently as much in terms of computational efficiency (making problem solving more tractable) as in terms of data modeling/interpretability.

Dependencies inside data are synonymous with redundancies and efficiency loss from an analysis viewpoint. Detecting and overcoming these dependencies is typically performed by means of a statistical latent factor analysis (using PCA, PCR, ICA, ICR, Bayes factors, etc. depending on application requirements). Refer to the valuable reference suggested by Umar Niazi (available from https://www-bcf.usc.edu/~gareth/ISL/ISLR%20First%20Printing.pdf).

Once a subset composed of the major uncorrelated/independent/discriminatory latent factors (statistical features) is identified, then a topological data analysis (TDA) may follow up for extracting now on an invariant, persistent and robust subset of so-called topological critical points (topological features). These critical points characterize the intrinsic signature of the data.

As an additional piece of information, TDA is not incompatible with information theory and statistics. Follow:

• B. Michel, " A Statistical Approach to Topological Data Analysis ", 2015 - https://tel.archives-ouvertes.fr/tel-01235080/document
• Edelsbrunner et al., " Topological Data Analysis in Information Space ", 2019 - https://arxiv.org/pdf/1903.08510.pdf

Better to combine Feature selection along with PCA. Combining these two techniques you will get better classification accuracy.

1. https://www.quora.com/What-is-the-difference-between-principal-component-analysis-PCA-and-feature-selection-in-machine-learning-Is-PCA-a-means-of-feature-selection

2. https://stats.stackexchange.com/questions/182711/principal-component-analysis-vs-feature-selection

3. https://stackoverflow.com/questions/16249625/difference-between-pca-principal-component-analysis-and-feature-selection

Thank you Dr. Chiel Bakkeren for sharing your experience. Thank you Dr. Umar Niazi , Dr. Stéphane Breton and, Dr. Dr R Senthilkumar for good references.

I have worked in past over this my observation was Pca blindly takes any feature vector without knowing what classification error it has only highest eigen value is considered ,this is the biggest drawback which has been proved in my work .Pca always perform better with combination comparatively along with Kernal svm ,like Naive Bayes classifier ,Dimension reduction comes naturally to it it is classical approach for dimension reduction mathematically proved we still have no challenges over that part