Ibrahim Niftiyev , could you provide more information. How much variance does each factor explain?

Most people set the eigenvalue criterion to 1, but it means a factor explains as much as an original item and that is not much of a data reduction.

If the variance of the factors are low, you could also perform an ANACOVA using the original seven items. Or do a CART analysis to find the most important item(s) among the seven.

Thank you, dear Peter Moorer

I guess the screenshot below answers your question about the variance.

Thank you for your recommendations

Best

Ibrahim

Could you provide the factor loadings as well?

You might be able to force a one factor solution. 4.7 means the information of almost 5 items is preserved. Do you have a pattern in the matrix of residual correlations? Did you try a one-factor solution with ML as estimation procedure (gives the fit of the model).

I haven't applied any estimation techniques so far. Only PCA analysis.

Ibrahim Niftiyev , could it be that Fr_Li is coded wrongly? Maybe a negatively formulated item? Or something else that could explain why this item loads low and negavtivly on fac 1, while the other load high and positive. Item 1 loads on both factors relatively high.

Peter Moorer, the variable Fr_LI is based on the official statistics on labor input in the fruits sector. My guess, it this variable has outliers that create structural breaks in time series.

Ibrahim Niftiyev , does it include seasonal workers as well? That might indeed create structural breaks. It might be better to take all variable as independent for your dependent item. Do you have time series or cross sectional data? With cross sectional data, you might want to try data mining using regression trees.

I have time-series data. thank you for your valuable comments and for your time dear Peter Moorer

Best.

Ibrahim.

Dear Ibrahim Niftiyev ,

While I fully agree with insights shared by Peter Moorer , I have a few enquiries or questions and points to make.

Since PCA can be used for dimensionality reduction, ranking, regression, and clustering, may you please advise what your major final reason(s) for conducting PCA is(are) in your case? In addition, PCA is a large sample procedure in which observed variables are standardized, e.g. mean=0, and SD = 1, diagonals of the matrix = 1. In light of this background, please indicate the sample size of your dataset? Lastly, could you please provide full names of your variables and indicate measurement scales of the variables (noting that PCA is applicable when using measurement scales are interval or ratio levels). Your responses would guide us to suggest possible informed answers to the primary question you raised regarding whether you can use the index as an independent variable in simple or multivariate (multiple?) regression.

Looking at the loadings of the variables in the Component Matrix, the variable FR_PROF seems to be demonstrating a complex structure by the mere reason that it has high scores/loadings in both Component 1 (FR_PROF loading = 0.724) and Component 2 (FR_PROF loading = 0.574). A similar or same picture is shown in the Pattern Matrix in which you applied the Oblimin with Keiser Normalization criterion under the Oblique rotation method which assumes that components are correlated – commonly highly recommended! As I mentioned earlier, in the Pattern Matrix after rotation, FR_PROF is still exhibiting a complex structure shown by fairly closely high loadings in both Component 1 (FR_PROF loading = 0.645) and Component 2 (FR_PROF loading = 0.621). In the same manner, the variable FR_CULT has a loading = 0.774 under Component 1 and a loading = 0.414 under Component 2 – which is close to 0.774. Could you please advise the level you have set your minimum threshold at?

Given the complex structure (ie. variable FR_PROF loading highly on more than one Component) identified as indicated above, please note that the final solution or overall results of your PCA need to or should satisfy the following common interpretability criteria:

(a). rotated component pattern should demonstrate “simple structure” – ie. each of the variable should have a high loading on one component and near-zero scores/loadings on the other components,

(b). there should be at least three variables with significant scores/loadings on each retained component.

Regarding your question of whether on not to perform simple or multiple regression, since PCA is much suitable when there are correlated variables whose number should be reduced to explain the same amount of variance with fewer variables (principal components), the presence of collinearity between independent variables is usually detrimental to multiple linear regression models derived by OLS, hence collinearity should be minimized as much close to none as possible – to further avoid a possible significant increase in the variance and instability of coefficients.

Looking forward to hearing from you!

All the best.

Dear Elvis Munyaradzi Ganyaupfu

My main goal to apply PCA analysis is to evaluate the sub-sectoral economic performance of vegetable and fruit sectors by combining 7 original variables into one. Said another way, I have 7 same category indicators for vegetable and fruit sectors. The sample size of my data set is 26. You can see the full description of my variables from the screenshot below.

What do you mean by minimum threshold?

Thank you for your detailed explanations and I am sorry for my late response. I am new to PCA analysis, so there are a lot of gaps in my knowledge.

Best

Ibrahim

Dear Ibrahim Niftiyev ,

Thank you for the response and information shared on your PCA analysis – quite helpful.

Sincere apologies if I might have used the term “threshold” loosely. What I mean by “threshold” is the minimum score/loading a variable should have under a particular component. In other words, it is the minimum loading or score size which you have set as a criterion to regard and select a variable as significant for classifying a variable under a particular Component. For Instance, results in your Pattern Matric show that scores of variable FR_CULT as 0.774 under Component 1 and 0.414 under Component 2, so if you have set your minimum score size at 0.5 to regard a variable as significant, then you classify FR_CULT under Component 1. Using the same criterion of 0.5 as the minimum score, the variable FR_PROF loading with scores of 0.645 under Component 1 and 0.621 under Component 2 suggest that FR_PROF has a complex structure since it is loading with high scores in more than 1 component, and therefore the respective variables should be eliminated – unless a different analytical procedure is applied.

To probably reach at a conclusion on your analysis, I have a few suggestions:

(1). Could you please first compute and share results of the correlations between the variables. Correlations between variables should as much low as possible to avoid the potential problem of multicollinearity (Steven, 2002).

(2). Your Scree Plot shows that the number of components was extracted based on the Eigenvalues greater than 1 – thanks. But please advise how you determined your Maximum Iterations for Convergence – by default at 25 or you did input a pre-determined number? Such information can help understand the complete procedure you adopted in the determination of the number of components in your PCA. Probably you can read Huang and Tseng (1992) on the procedure for determining the number of components in PCA – full reference herein below.

(3). If your data is time-series, please assess and share the distributions of each of your variables.

(4). Sample size as one of the factors that influences the accuracy of procedures in PCA (Velicer, Eaton & Fava, 2000). I am NOT implying that your sample size is small, but I am not sure if the sample size of 26 relative to the number of variables (7) that you did run PCA is good enough, – you can read Velicer, Eaton & Fava (2000) with full reference provided below.

(5). Variable-to-component ratio (v:c) is an important feature in evaluation of PCA results. In your Pattern Matrix results, despite the complex structure demonstrated by FR_PROF, the distribution of number of variables in Component 2 relative to Component 1 raises some questions. The number of variables per component (which is measured counting the number of variables correlated with each component in the population conditions) influences the accuracy of the results, with more variables per component producing more stable results. If we exclude FR_PORF (due to complex structure) from your results, Component 1 will contain 5 variables while Component 2 will contain only one variable (FR_LI).

Suggested Readings:

Huang, D-Y. and Tseng, S-T. (1992). A decision procedure for determining the number of components in principal component analysis. J. Statist. Plan. Inf., 30, 63–71.

Stevens, J. (2002). Applied multivariate statistics for the social sciences. Mahwah. NJ: Lawrence Erlbaum Associates.

Velicer, F. W., Eaton, C. A., & Fava, J. L. (2000). Construct Explication through factor or component analysis: A review and evaluation of alternative procedures for determining the number of factors or components. In R. D. Goffin, & E. Helmes (Eds.), Problems and solutions in human assessment (pp. 42 -71). Boston: Kluwer Academic Publishers.