Hi Phein-Chin.

Instead of giving what I think, I believe you should read some methods research to find and form your own understanding. That is what worked for me. Just one thing; regardless of authors' takes, I'd say there is a common agreement that PCA has a specific application (item reduction) and that it is not a true factor analysis method because of its statistic and psychometric assumptions. Another pointer, pay attention to rotation methods as there is also discussion regarding orthogonal vs oblique approaches. I studied these references for my work and they helped me a lot, especially the first one:

Costello, A., & Osborne, J. (2005). Best practices in exploratory factor analysis: Four recommendations for getting the most from your analysis. Practical Assessment, Research & Evaluation, 10(7), 1-9. Retrieved from http://pareonline.net/getvn.asp?v=10&n=7

Thompson, B., & Vidal-Brown, S. (2001). Principal components versus principle axis factors: When will we ever learn? Paper presented at the Annual Meeting of the Southwest Research Association, New Orleans, USA.

Fabrigar, L., Wegener, D., McCallum, R., & Strahan, E. (1999). Evaluating the use of exploratory factor analysis in psychological research. Psychological Methods, 4(3), 272-299.

DeCoster, J. (1998). Overview of factor analysis Retrieved from http://www.stat-help.com/notes.html

If you have problems in finding any of these articles, tell me and I will send it/them to you.

Best wishes for your research :)

Hi Phein-Chin.

Instead of giving what I think, I believe you should read some methods research to find and form your own understanding. That is what worked for me. Just one thing; regardless of authors' takes, I'd say there is a common agreement that PCA has a specific application (item reduction) and that it is not a true factor analysis method because of its statistic and psychometric assumptions. Another pointer, pay attention to rotation methods as there is also discussion regarding orthogonal vs oblique approaches. I studied these references for my work and they helped me a lot, especially the first one:

Costello, A., & Osborne, J. (2005). Best practices in exploratory factor analysis: Four recommendations for getting the most from your analysis. Practical Assessment, Research & Evaluation, 10(7), 1-9. Retrieved from http://pareonline.net/getvn.asp?v=10&n=7

Thompson, B., & Vidal-Brown, S. (2001). Principal components versus principle axis factors: When will we ever learn? Paper presented at the Annual Meeting of the Southwest Research Association, New Orleans, USA.

Fabrigar, L., Wegener, D., McCallum, R., & Strahan, E. (1999). Evaluating the use of exploratory factor analysis in psychological research. Psychological Methods, 4(3), 272-299.

DeCoster, J. (1998). Overview of factor analysis Retrieved from http://www.stat-help.com/notes.html

If you have problems in finding any of these articles, tell me and I will send it/them to you.

Best wishes for your research :)

Hi Lim,

Optimal Discriminant Analysis (ODA) and the related classification tree analysis (CTA) will help you to find the best (the most accurate forecasting) combination of independent variables (determining factors).

This book by Paul R. Yarnold will help you:

http://www.amazon.com/Optimal-Data-Analysis-Guidebook-Software/dp/1557989818/ref=sr_1_3?ie=UTF8&qid=1417517745&sr=8-3&keywords=Paul+R.+Yarnold

The book is easy to follow, coming with the software to conduct basic UniODA.

He also has many papers on the topic available on RG and his own website: http://odajournal.com/

Richard

Quite often the number of retained factors is based on whether you sre able or not to identify a meaningful "label" for a factor. Stop when you don't know anymore what a factor means.

I totally agree with Raid's practical opinion. I think Structural Equation Model (SEM) might also help you.

You can also use Fruckter formula to identify number of factors.It mainly depend upon the number of variables that you used.

Hi,

Informal method: Check out the percentage of the total variance accounted, as you do in PCA.

Formal method: Use the likelihood ratio test (LRT) by assuming normality. The default function "factanal" in R can be simply used to perform the LRT.

Good luck!

Hi,

PA it's not only for PCA - there is program for FA (see: Schmitt, 2011). There is also MAP method (minimum average partial correlations - see: Velicer, 1976) - but it is mostly for PCA. You can also use model fit indices (see: Preacher, Zhang, Kim, & Mels, 2013).

The most important criterium is to identify a meaningful "label" for a factor

Velicer, W. F. (1976). Determining the number of components from the matrix of partial correlations. Psychometrika, 41(3), 321–327. doi:10.1007/BF02293557

Schmitt, T. A. (2011). Current Methodological Considerations in Exploratory and Confirmatory Factor Analysis. Journal of Psychoeducational Assessment, 29(4), 304–321. doi:10.1177/0734282911406653

Preacher, K. J., Zhang, G., Kim, C., & Mels, G. (2013). Choosing the Optimal Number of Factors in Exploratory Factor Analysis: A Model Selection Perspective. Multivariate Behavioral Research, 48(1), 28–56. doi:10.1080/00273171.2012.710386

see also:

Thompson, B. (2004). Exploratory and confirmatory factor analysis: understanding concepts and applications (1st ed.). Washington, DC: American Psychological Association.

What a very good question. All the responses provide a different perspective Principal Components Analysis (PCA) is only used for a correlation matrix that is singular - so no you can not treat them as analogous. You seem to have a good handle on using Eigen Values to select factors - I would suggest that in the selection of variables you go back to the Bradford Hill criteria and your research construct - does it make sense? Is there a plausible explanation consistent with one variable over the other? This is one reason why we use focus groups prior to finalizing our experimental design. The issue with PCA is not only to determine the principle components that make up the vast amount of variance but to remove those that are covariance. The issue with parallel analysis is that it is heavily dependent on the assumption on normality of the variables. The assumption of PCA is that you desire to conduct a PCA of your data using the principal components - right? Wrong - the PCA is to identify those variables that explain most of the variance so that you can have a parsimonious set of (and perhaps consider interactions) variables (and choices of which variables to drop that will be most consistent with the carefully considered research construct.

To all,

Thank you very much for the valuable comments (and reading suggestion)! Learn something new - Bradford HIll criteria, Simpson's paradox etc

This question is initially asked as all my variables are continuous and no respondents involved as variables are extracted from images. One of the objective is to explore whether certain variables group together to form a general concept. Hence, EFA is preferred over PCA. My confusion is most probably due to the use of SPSS since same procedure is followed for conducting PCA or FA, also due to reading. 

Came across ViSta - The Visual Statistics System by Forrest W. Young which provided PA, PCA and FA as separate module. Anyone have experience using this application?

Number of factors extracted on the basis of % of variance explained. what level of variance is satisfactory depend on the type of problem. Normally, in marketing research, it is recommended that the factors extracted should account for at least 60% of the variance.

I would say that the in addition to the scree plot and the eigenvalue greater than 1 rules it is very important that the factor you extract is meaningful in terms of describing the concept you are trying to measure. I also feel that the percentage of variance explained should not be too small

1. Keep as many factors as you can identify as being meaningful.

2. Keep as many factors as there are eigenvalues that exceed 1.

3. Make use of a scree plot to assist you in (1) and (2).

4. The final choice of factors should explain a sufficiently large portion of the variability.

These are the main four rules of thumb that several people have suggested in this forum.

Dear Phei-Chin,

Some researchers also limit the number of factors to be retained based on factor loadings. In some cases, prior research in the discipline/topic may help us to set this cut off point.This may be greater than 0.4 or 0.5 depending on the specific area of interest.

The issue of the number of items to be retained is quite different from the number of factors to be extracted.  As Abhay explained you can decide the number of factors to be exctracted based on the percentage of variance explained by the model. For instance, if you want to get 60% of the variance explained by the model and if four factors explain this percentage of variance, you can limit the number of factors to be extracted to 4.

On the other hand, for items to be retained or not the major issue is their factor loadings (item-total correlation).

I routinely use Watkins' (2000) program to conduct PA to determine number of factors to extract in PAF/EFA . I find it tends to be a little more conservative when compared to other methods. I get your concern about the fact that the simulated random eigenvalues are obtained via PCA and the fact that PCA and FA are not equivalent. O' Connor (2000) has syntax for PA in SPSS though I don't know whether it obtains estimates from PCA or FA. The PA function in the PSYCH package provides the option to simulate random values from both PCA or FA although I tend to find the FA derived values tend to over-factor (e.g., 8 variables, 6 factors suggested).

In general, I always recommend PA and MAP as a compliment to the visual scree test for determining the number of factors to extract. If results diverge, it is typicaly recommended to over rather under extract (Wood, Tataryn, & Gorsuch, 1996). Run all the extractions suggested and examine the pattern and strength of the loading's across the models and retain the model with the best theoretical and statistical convergence.