I have 8 response variables in different scales. The variance of 4 variables are very high and 4 variables are low. The all observation are separated in 3 groups. By MANOVA, there is a significant effect of grouping. Which correlation matrix should be used in PCA in this situation?
Dear Rezaul,
My default method is to scale each variable to unit variance since that put them on equal footing. If you don't scale them, the variables with large variance is given more weight in the PCA model. This can be correct in some cases, such as for spectral data, but if you mix variables from very diverse sources what unit you use will make a difference.
When it comes to defining 'best in PCA' there is no simple answer. Just because one PCA model explains more variance doesn't mean its better. I tend to see which model gives a biological relevant results.
Hope this helps!
As Anders points out, scaling variables from diverse sources is important. Correlation matrix is less sensitive to scaling, which is why the variation explained is lower, but this in itself does not indicate better or worse. My preferred method to use is whatever one is more interpretable. Pre-scaling your data means you understand exactly what transformations it has been through. This should make it easier to tie the results back to the original data. As well as unit variance scaling, consider mean centering if the different variables cluster around different mean values - but if using both always do mean centering first.
Dear Rezaul, I am not totally agree with previous comments. First of all I should say that using correlation matrix standardize the data automatically, and you don't need to transform them using Z score. In other words correlation matrix is not affected by the scales. This problem that you have encountered is due to a little nonlinear correlation between your variables. So, if the data standardized for correlation matrix, the curvature gets more obvious. Because the range is reduced highly enough, but the pattern exist anyway.
I recommend to test the nonlinear correlations and remove it first.
I agree with the first two answers (Anders and James). My default is to use the correlation matrix which makes all variables equally important in PCA.
It depends what you want to do. If you want to understand/model which variables vary together, PCA of the correlation matrix will be helpful. If you want to approximate the major variability of your data in a few variables (for instance, clean up a noising image or signal), PCA of the covariance matrix will be helpful.
What PCA does is identify the direction (or weighted combination of variables) with MAXIMUM variation. You mention that you have four variables with a large scale. Variables with huge values will typically dominate the total variation of a dataset as you observed using PCA on the covariance matrix.
If your data contains strictly positive measurements of quantities like weight, length, or price, you might want to apply PCA to the log of your data values.
I just re-read the title of your question, and noticed your reference to "partial correlation". If you are interested in partial correlation, you should work with the precision matrix, which is the inverse of the correlation matrix. If the data is normally distributed, then zeros (or near zeros) in the precision matrix imply conditional independence of the variables. See: Lauritzen, Steffen L. Graphical models. Oxford University Press, 1996.
- Badges
- Science method
- Similar topics
- Biological Science
- Ecology