Hi Chetan,

Is it a principal components analysis or is it the principal components method of factoring in a factor analysis that you will be using? 

Hi Chetan,

Better to do on principal component analysis.

Thank you

Principal component vectors capture directions of maximum variance in your N dimensional vector space.

You can pick the top 'k' PCs to explain a certain percentage (say 90%) of your data variance.

This helps reduce dimensions and also provides some immunity to noise.

Congrats for floating a useful issue. I agree with Prof Vikas Ramachandra. The PC vectors having the maximum values (as provided by the analysis into a number of new factors) should be identified and you've to relabel them. I've come across a number of studies/ papers that have failed to relabel them. Relabelling those factors is indispensable to give rise to a new set of factors. All the best 

You might find it helpful to take a course on factor analysis to get a more complete understanding that you will probably get here.  Notwithstanding, there are a few standard things that you will want to know.  Making decisions about what to do with them or how to interpret them is sometimes controversial and may depend on both substantive and statistical judgments. 

Assuming that you are doing a principal components analysis, you may need a reason for doing it.  Are you trying to find a decomposition of your correlation matrix that accounts for all the variance in the system?  If so, you would want to keep all of the components.  If not, then you have to decide what to do about communalities, the variance or reliability of each variable with itself.  These values usually replace the diagonal of the correlation matrix and becomes one of the major differentiating features of factor analysis from component analysis.  Then you will want to perform some kind of decomposition.  There are many algorithms from simple eigen decomposition to maximum likelihood methods for performing this decomposition.  The overall goal regardless of how it is computed is to derive a structure where variables load (are regressed) onto latent variables that we might call factors or components.  Under most situations this is an eigen decomposition where eigenvectors are the factor loadings from some perspective in n-dimensional space and the eigen value associated with that eigen vector represents the variance accounted for by that factor. 

The next task is usually to determine how many factors or components you wish to keep to explain the amount of variability in the system.  If you consider components (eigen decomposition of a correlation matrix) you can plot the eigen value on the y-axis and the component number (yes, it is ordinal) on the x-axis.  You will probably notice that there is a sharp elbow in the curve shown by that plot (called a scree plot.)  The factors with the largest eigenvalues up to and including the elbow represent the number of factors you would usually retain.  It is based on the idea that at some point you reach diminishing returns and keep only those factors that continue to account for signal, and not noise in the correlations.  Another way to determine the number of components is to select all eigen vectors whose eigenvalue is greater than 1.0.  This means that the component or factor accounts for as much variance as would be expected by at least one variable.  Note, in this case the sum of the eigenvalues equals the number of variables in your correlation matrix. 

The next task is to rotate your factor matrix to something called “simple structure”, which involves finding a position to view the projections of the data onto your factors that maximizes some of the variables loading on factors and simultaneously minimizes their loading on other factors.  There are many strategies for rotation, some that constrain factors to be orthogonal (independent) and oblique (correlated.)  Some methods are Procrustean in that you will rotate the factor matrix to get as close to an a priori structure. 

This rotated matrix is the one you will want to look lovingly at because it represents the “best” view of your data in the reduced space.  Whether you will be looking across columns or down rows is a function of the extraction method and the rotation method.  Often each factor is named to represent the variables that load on it. 

I realize that this is a very terse explanation of factor analysis and component analysis.  It will not make you an instant expert.  Hopefully, it will provide enough conceptual background for you to read a simple text on factor analysis such as  R. J. Rummel (1970) Applied factor Analysis.  If you are really interested in getting into this in some depth, then let me recommend Stanley Muliak (2009) Foundations of Factor Analysis as a good starting point.  You may find some useful interpretive guides in the SPSS manuals or on the internet. 

You may also want to investigate confirmatory factor analysis and structural equation modeling, i.e., LISREL. 

Best wishes.

David Drehmer

Dear David and Vikas, 

Thank you for your comments and suggestions. As put forth by David, yes I need to do some more in-depth study to get what I want from PCA.

I ran my Data for 23 factors, with Varimax rotation. To check the appropriateness,  Kaiser-Meyer-Olkin (KMO) statistic was also used. A scree plot was also generated.

From here, all the factors in the rotated component matrix (SPSS) were checked, but the results were not what most tutorial pointed to. 

Hence the confusion. I just have the out put files (in word and pdf) with me now, and I cant run the data again, so wanted to know, how to check what went wrong, or why my result is not what it should be. 

Is it because there was a deficiency or fraud in the data?

I will read through the texts mentioned by David.

If any one wants to have a look at the output values, I have a copy of them in word/pdf. Can send the same.

Thanks All. Much Appreciated. 

Regards, 

Chetan 

Chetan,

23 factors seems mind boggling in my world.  Let me ask you, how many variables did you have in the analysis?  How many subjects?  Did you have a missing value issue, if so, how were those missing values handled?  It would be unreasonable to diagnose without actually seeing the analysis and knowing something about the substantive area in which you are working.  If you would like to post (attach as a file) your word or PDF output, I, and perhaps others, would be willing to take a look at it to see how we can help.   

Factor analysis is usually  carried out as a data reduction technique and test of the validity of the instrument.  Two statistical tests are  conducted in order to determine the suitability of factor analysis. First, the Kaisers-Meyer-Olkin (KMO) measure of sampling adequacy score  and it should be  well above the recommended level of 0.50. Second, the Bartless test of sphericity and should be significant (Chi Square with  P < 0.00), indicating that there are adequate inter-correlations between the items which allow the use of factor analysis.  Principal axis factoring is  used as an extraction method. But  Principal axis factoring   is not a complete solution and it is not the desired clean solution. Therefore it should be followed by  an oblique rotation as the  rotation method   for better interpretation    using Eigenvalue greater than one criterion.   Moreover, principal component analysis is used in Canonical Correlation analysis and that case you interpret the end product.

If you want to then use the factor scores for a follow-up statistical analysis in which the factor scores are the dependent variables with some (new) variables, make sure to use  a multivariate analysis if the factor rotation method used was oblique since the factors then lose their independence from each other. If an orthogonal rotation is used, it is fine toi use univariate analyses with the factor scores.

 My suggestion: https://www.researchgate.net/publication/319469038_New_Interpretation_of_Principal_Components_Analysis

Article New Interpretation of Principal Components Analysis

 PCA  Is incomplete  solution for the construct validity of the instrument.   As a data reduction technique you have to use a clean solution such as oblique solution  and a criteria for the factor loading such as 0.70. The dimensions will tell you how many dimensions the instrument is measuring and the factor loading will tell you the relation between the statement and the dimension. What's more, it is recommended to compute factor scores for further multivariate analysis.

I totally Agree with Professor Abdulrazzak Charbaji

One more point; have you standardized the variables used in the PCA? Variables with larger values will otherwise dominate the analysis.

Dear Dr. Charbaji,

I have used varimax rotation and also KMO for validity. Let me know if I've gone wrong??

Dear Dr. Amin,

Would I still required standardization if I used a 7-point likert scale. There are no change in measurement units here???

Thank you for your time

Hi Chetan,

If all variables used in the PCA are measured on the same scale, there is no need for a standardization.