When the number of input variables used in neural network is high and we want to reduce the number of inputs of the model PCA is an appropriate method. But there is a question about PCA. Is it necessary to normalize data before performing principle component analysis?
Hello !
Yes, it is necessary to normalize data before performing PCA.
The PCA calculates a new projection of your data set. And the new axis are based on the standard deviation of your variables. So a variable with a high standard deviation will have a higher weight for the calculation of axis than a variable with a low standard deviation. If you normalize your data, all variables have the same standard deviation, thus all variables have the same weight and your PCA calculates relevant axis.
I hope that this message can help you !
I will add little more for the Lucile's explanation....
As different variables in your data set may be having different units of measurement. Eg: one may be cost, other production in numbers, another percentage interest in purchase etc. It is necessary to normalise data to get a reasonable covarianc analysis among all such variables. Another example to appreciate the normalisation is analysis of students data in a class room may include...height and weight and establishing a correlation between them.
By correlating the matrix you are doing the the same normalization, what we mean in above paragraphs.
Dear Darsi Piyathilake, generally log (variables) are considered for very high magnitude variables. Suppose, you are having 5 variables, 4 are few tens in magnitute, where as the other variable is in the order of 4 or 5, where the magnitute of 5th variable is very much off from the rest, you can better consider log(variable for the fifth). But in your problem, If i understand correctly, you have compositional ratios as your variable, means they are less than one, which log(compositional variables) become negative. In this case, the scattering of these new variables in loadings plot may give you some information, regarding its usage. You can compare the distribution of scores and loadings of with and without logarithmic variables, and depending on your problem of concern, you have to decide, which best suites for your application and experimental conditions.
Can anyone also tell how the normalization is carried in general practice of EOF analysis? I know the data are divided by the standard deviation but what is the standard deviation used? Is it time standard deviation at different data (grid) points?
Dear Sree,
please guide me about PCA In eviews application.
I am using EVIEWS Software, and in the attached figure in the computation box you can see the correlation circled option. in this case do we need to normalize the data or it is done automatically. please explain. thanks
In addition to the above, I also recommend reading the following links.
https://www.researchgate.net/post/In_which_case_data_need_to_be_normalized_before_PCA_Cluster_analysis
https://stats.stackexchange.com/questions/69157/why-do-we-need-to-normalize-data-before-principal-component-analysis-pca/69159
https://www.quora.com/Why-is-it-beneficial-to-center-and-normalize-the-data-before-running-Principal-Component-Analysis-on-it
This can be useful https://www.reneshbedre.com/blog/principal-component-analysis.html
Sree Rama Murthy A. 's answers make perfect sense! Thank you. However, I'd like to add a note - it's important to understand normalization/scaling/standardizing since these terms are used quite loosely. The use of normalization in some of the above posts really mean standardizing the data with mean 0 and unit variance. This should be helpful: https://towardsdatascience.com/scale-standardize-or-normalize-with-scikit-learn-6ccc7d176a02
Sree Rama Murthy A. Hi. I am working on a dataset containing more than 100 features. Most of these features are anthropometric measurements such as waist circumference, hip circumference, height, weight etc.
I have selected 20 of these features based on the problem I am working on and now I want to use PCA to find the most important features among these 20 features and come to the maximum of 8 features as PCA is one method for feature selection in clustering analysis (it is an unsupervised task).
I have applied the Shapiro-Wilk test on all these features and I found that they are not distributed normally as the p-value for all of them is more than 0.05 and there is some skewness in the right tail.
My question is what pre-processing or feature engineering steps I should apply to my dataset before applying PCA on these 20 features and is scaling the dataset with StandardScaler() the only thing I should apply on these skewed features? Thank you so much.
Soheila Ahmadi, probably this discussion will help you.
https://www.researchgate.net/post/What-is-the-best-way-to-scale-parameters-before-running-a-Principal-Component-Analysis-PCA
It is important to consider the nature of the dataset when deciding on the preprocessing steps before performing PCA. If the different features in the raw data have varying types, such as temperature, moisture, and vibration, with different units and magnitudes, normalization is necessary to account for these differences. However, if the features have the same type, such as multiple vibration measurements acquired from different sensors with the same unit, it may be beneficial to allow a more sensitive feature to have a dominant influence in the PCA process.
Nevertheless, simply subtracting each feature from its average (mean centering) may be beneficial regardless of the type or nature of the features. Mean centering helps to remove any bias and ensures that the principal components are based on the variations in the data.
- Badges
- Science topic