So here is the thing:
- I ran a Likert scale questionnaire that had 16 questions (n=328) and afterwards wondered if these questions 'overlapped' somehow. Therefore, I generated a correlation matrix and ran KMO and Bartlett's Tests aiming to see if a Principal Component Analysis (PCA) would be a good fit for my data and what I had in mind. The KMO and Bartlett tests came out favorably. The correlation matrix turned up to have no correlation above .6 but the number of correlations in the .1 to .5 range outnumber the ones between 0 and .1.
- So, based on these results, I ran my PCA (varimax rotation) and got back 4 components with Eigenvalues above 1. The thing is, that these only accounted for 51% of the cumulative % of variance explained.
- My next step was to try something else, I ran a new PCA this time asking for 8 components (since this would put me above the 70% of cumulative variance explained range). Now the problem I had this time was that I still had some cross-loading going on (four cases of secondary loadings above .4) and, additionally, I still couldn't really make sense of some of the components it offered me.
- I kept going on experimenting with PCA's with more components and finally got an outcome with almost no cross-loadings and that I could make sense of when I asked for 11 components.
Now here is my question: given the high number of components that I had to ask for in order to achieve this outcome, is my 11 component PCA still reliable? Or should I leave it out of my research?
*BONUS more subjective (and hard) question:*
Let's say my PCA groups two variables into one same component and without cross-loading. Then, when I look at my original correlation matrix (not the PCA matrix's) I see that these two variables correlation is, in fact, only moderate at best. Now my question: Given this scenario, is it even fair to try to draw conclusions and deliberate about the possible reasons for these variable's interrelations? Because what it would seem to me is that the PCA component is telling me one thing and the original correlation matrix something else.
Common factor analysis and principal components / composite based methods have diferent strengths. Common factor analysis is better for accounting for the covariances among observed variables while principal components is better for accounting for the variances of the observed variables (see the attached chapter by Karl Joreskog). The point is that principal components and the strength of correlations don't have that much to do with each other.
Useful questions, Leônidas. You're not doing anything wrong. In fact, you're doing your analysis very well.
It's not about "reliability" but about validity and interpretability, and simple usefulness.
11 factors from 16 variables isn't a good improvement on your original 16. More than that, if you plan to use your 16 questions in any further research, in another questionnaire with a different population, then you'll need to reconstruct those factors - and factor loadings will always be different. Or, like most social researchers, you'll ignore the small loadings of most variables on each factor, and generate new factors which are the sum, or average, of the small number of high-loading variables on each factor. Guess what happens when you do that... you get new variables (based on those factor solutions) that are somewhat correlated. Usually not highly correlated, but probably they're just as correlated as your original 16 questions were in the first place.
Rather than feel concerned, you should congratulate yourself: You asked 16 questions that, I'm guessing, were designed to measure distinctly different (uncorrelated) opinions, and you succeeded.
On the other hand, it's also possible that most of the responses to all 16 questions are simply random. But that's only likely if you're using a sample of undergraduate students who, I've discovered, see such pilot study questionnaires as not just an opportunity to corrupt their professor's work, but a responsibility!
The idea with PCA is to reduce the data and not to extend it, With 11 components the data is not reduced in an important way according to my opinion. Do a scree plot and see where a natural break is indicating the vital number of components.
Before you make any decisions, I suggest you re-run your factor analysis with Principal Factors and oblique (correlated factors) rotation. Both these methods make more reasonable assumptions for working with social science data.
Also, eigenvalues greater than zero is a relatively arbitrary basis for choosing the number of factors, so I agree that you should examine a scree plot as an alternative. If you have the option, a "parallel" analysis is a useful addition to a scree plot.
Thank you all, once again. Based on your answers and the current circumstances, I'll probably drop my PCA and go one with my analysis without it.
Hume F. Winzar, thank you, this was really insightful. Still, if I may, I would like to know your take on my second question (just for further reference). When a PCA component groups 2 variables that the original Correlation Matrix did not show as with strong correlation, does this give me grounds to discuss about the possible connections between each other? To put this into context let's say that my PCA grouped a variable regarding "(software) being developed for passion (instead of for monetary gains)" and another regarding "creative freedom" despite only a moderate-low correlation between both in the original Correlation Matrix (not the PCA's component and rotated component matrix). Is it fair to write down the assumption/conjecture that "the perception of creative freedom is related to software being developed for passion"? I ask this because, as I see it, the PCA component would be telling me one thing and the original correlation matrix something else. However, it is also possible that what I'm asking here is totally outside of the scope/purpose of what the PCA and correlation matrix actually do and show me (I wouldn't be surprised if that were the case, heh). Thank you.
s. Béatrice Marianne Ewalds-Kvist, the scree plot criteria also put me at 4 components, therefore, unfortunately, incurring in the same problem I stated originally in my the question : / Thank you for your advice, though.
David L Morgan, sorry if I was unclear but just to clarify, I did not use eigenvalues greater than zero as my criteria for choosing the number of component. I initially used eigenvalues above 1, and later total cumulative variance explained above 70%. Although I agree that I can't really explain in a more... "rule-based way"... why I would work with 11 components instead of 8 (since both would be above the 70% range). The fact is that 11 solved the cross-loading and allowed me to understand the components while 8 did not. Still, thank you for your advice, I will certainly take it into consideration!
Thanks for following up, Leônidas Soares Pereira . I'm sorry I didn't address your second question. It's a good question, and one that made me think quite a lot when I was first using EFA, a very long time ago.
The matrix algebra explanation simply doesn't help an ordinary person like me. I prefer to think about vectors and angles - even tug-boats pulling a big ship at different angles. Yes, seriously.
Two insights that helped me a lot:
So, consider a correlation between two variables of, say, 0.40. That's not much. The angle between the two variables would be 66 degrees (Arccosine of 0.4 = 66). Draw two lines with equal length at a 66 degrees angle, and label them Variables A and B respectively. Pretend that they're tug-boats pulling a ship. Now a PC is the resolution of the two vectors; the direction that the ship will move because of the tug-boats. Draw that line in a different colour, and call it Factor_1. If the vectors are the same strength (1) then the Factor should run exactly in the middle of the two vectors. The angle between Variable_A and Factor_1 will be 33 degrees. And the angle between Variable_B and Factor_1 will be 33 degrees. The Cosine of 33 degrees is 0.84. That's also the correlation between the Factor and each Variable is 0.84. That's relatively high!
So a Principal Component (a Factor if you're rotating it) that summarises two variables, that are not highly correlated with each other, can have an apparently high correlation between both those variables.
I hope that helps.
Dear Leônidas,
when you have items that might be more "ordinal" than "quantitative" (like a Likert score), Pearson's correlation might underestimate the correlation between the items. Try again your PCA by using a polychoric correlation instead (sometimes, with polychoric correlations, you get a non-positive-semidefinite matrix but usually it's not a big issue and you can force the matrix to be PSD).
To determine the number of components, instead of using Kaiser's criterion, try the parallel analysis (which gives you your real eigenvalues vs eigenvalues generated by means of a stochastic process - the idea is that you keep all the components which real eigenvalue is higher than the stochastic one).
Last but not least: you said you administered a questionnaire... are you sure that what you want to do is a PCA and not an EFA?
Cheers,
Marco
Very Sorry. I do not work anymore with quantative methods nor statiscs. Best regards
Again, thank you all for your help!
and special thanks to Hume F. Winzar , your explanation really REALLY helped me understand what I was actually looking at (metaphors help me a lot in understanding abstract subjects). I am very grateful!
Common factor analysis and principal components / composite based methods have diferent strengths. Common factor analysis is better for accounting for the covariances among observed variables while principal components is better for accounting for the variances of the observed variables (see the attached chapter by Karl Joreskog). The point is that principal components and the strength of correlations don't have that much to do with each other.
Why complicate things. Use 4 components as your scree plot tells you.
Try Oblique rotation method......
Article An Easy Approach to Exploratory Factor Analysis: Marketing P...
- Badges
- Science topic
- Similar topics
- Mathematics
- Statistics
- Statistical Software
- R