Hello Lorenzo,

First, if you are using a summated scale (scores on all pertinent items are summed to yield an overall (sub)scale score), then compute these for each case.

Then, dummy code country as two distinct values (e.g., 1, 2) for the two countries involved.

Finally, compute a correlation coefficient between the summated score and dummy variable for country. If you wanted to be safe, use a Spearman correlation. If you believe, like many apparently do, that a Likert-type summated scale score behaves sufficiently well as a pseudo-continuous variable, you could compute a Pearson correlation.

If you have only a single item (not a summated scale), then you should stick to the Spearman correlation.

Good luck with your work.

Along with what David Morse has recommended, if you are going to create a scale by summing your Likert-scored items, you should check the alpha coefficient for those items to determine the reliability of the scale.

Also, if you want to do the equivalent of a t-Test for each separate Likert-scored variable, the you can us the Mann Whitney rest.

Dear Lorenzo Izzo ,

There are two issues here:

David Morse 's answer to the former is a simple explanation of a complex issue. Please refer to my study guide on scale reliability with small samples. His suggestion of adding together values and using Pearson correlation may also not be the best approach. The regression scores from a Principal Component Analysis (PCA) are often preferable to adding values and you should first assess the distribution of your scatter plot before using Pearson correlation. Again, please refer to my study guide on Pearson correlation. The consequence is if the scatter plot is the wrong shape you should use Spearman correlation.

Finally, David L Morgan mentioned alpha scores. Again, there are many myths about Cronbach's alpha. PCA is considered by some to be a preferable technique, as explained in my first study guide.

Peter Samuels I am not sure why you are recommending Principal Component Analysis (PCA), when the more common preference with multiple indicators is Principal Factors Analysis (PFA) or its equivalent.

David L Morgan, I agree with your having reservations about the use of principal component analysis (PCA), but I wonder whether, by principal factors analysis as an alternative, you might be referring to exploratory factor analysis (EFA).

In fact, I confess (and publicly at that) to not knowing what what principal factors analysis is. Could you shed some light on that, please?

To my knowledge, there is a fundamental division between PCA and EFA (with the latter frequently recommended over the former), and within EFA there are a number of extraction methods - including principal axis factoring (PAF) and maximum likelihood (ML) - and it's necessary to draw appropriate distinctions, and "nesting" in relation to all of the above.

If my conceptions are incorrect, I'd be happy to be apprised of that.

Contributing my 2 cents, I would use neither PCA nor the proper Factor Analysis. since both of them require the input variables to be measures at interval level, whereas Likert-type response items are ordinal.

In fact, a good data analysis software like R would not even allow such variables into the FA procedure, unless they are recoded as dummy variables which is something very different.

And to answer the original question, there is of course no "correlation" between such variables, but it is possible to measure their association using a regular chi-square test.

Robert Trevethan: I think you are confusing what EFA is. In fact, PCA is exploratory because there is no expectation about how many factors will emerge, whereas the proper factor analysis (the confirmatory one) requires the researcher to specify the number of factors before running the analysis.

Adrian Dușa, I see we are in disagreement again - but that's OK if we create greater understanding for each other as well as for others who come to this thread.

I agree with you that PCA is exploratory, but my sense is still that there is a basic division between PCA and EFA. Furthermore, my impression is that both EFA and CFA are forms of factor analysis, with the former being exploratory and the latter confirmatory. So, to me, the primary distinction was between PCA and factor analysis, not between exploratory and confirmatory factor analysis.

I was basing my post on text within what I regard as authoritative sources such as the following:

Costello, A. B., & Osborne, J. W. (2005). Best practices in exploratory factor analysis: Four recommendations for getting the most from your analysis. Practical Assessment, Research & Evaluation, 10, 1–9.

Fabrigar, L. R., Wegener, D. T., MacCallum, R., C., & Strahan, E. J. (1999). Evaluating the use of exploratory factor analysis in psychological research. Psychological Methods, 4(3), 272—299. https://doi.org/10.1037/1082-989X.4.3.272

Gaskin, C. J., & Happell, B. (2014). On exploratory factor analysis: A review of recent evidence, an assessment of current practice, and recommendations for future use. International Journal of Nursing Studies, 51(3), 511–521. https://doi.org/10.1016/j.ijnurstu.2013.10.005

Matsunaga, M. (2010). How to factor-analyze your data right: Do’s don’ts, and how-to’s. International Journal of Psychological Research,3(1), 97–110. https://doi.org/10.21500/20112084.854

Returning to our interchange in another RG thread, I think it's interesting that very few researchers seem to get worried about factor analysis being based on what are regarded as ordinal-level data. On top of that, time and again they get results that are meaningful.

In relation to that, it's interesting that Hair et al. (2014; ref below) wrote that "indicators with ordinal responses of at least four response categories can be treated as interval, or at least as if the variables are continuous" (p. 612) and also that Norman (2010; ref below also) wrote that regarding ordinal data as if they are interval can “give the right answer even when assumptions are violated" (p. 627).

Hair, J. F., Jr., Black, W. C., Babin, B. J., & Anderson, R. E. (2014). Confirmatory factor analysis. In J. F. Hair, Jr., W. C. Black, B. J. Babin, & R. E. Anderson (Eds.), Multivariate data analysis (7th ed., pp. 599–638). Pearson.

Norman, G. (2010). Likert scales, levels of measurement and the “laws” of statistics. Advances in Health Sciences Education, 15(5), 625–632. https://doi.org/10.1007/s10459-010-9222-y

Where does that leave us? :-)

And I still don't know what principal factors analysis is . . .

Sorry to disagree (not looking for that) but in fact EFA is just a broad term for PCA (see Article Exploratory Factor Analysis: A Guide to Best Practice

Article Evaluating the Use of Exploratory Factor Analysis in Psychol...

The true distinction is indeed between EFA and (C)FA, and I would not equate EFA as a form of factor analysis. Data reduction techniques are something very different from testing hypotheses about the number of factors.

Adrian Dușa, I don't seek disagreement, either - only clarity.

You and I might be going in circles a little bit. The first item you cite in your most recent post (the article by Watkins) relies on information in the second item (the article by Fabrigar et al.) - and that second item is one of the items I included in my most recent post.

As I read the Fabrigar et al. article, there is, indeed, a difference between PCA and EFA - and that is in terms of computation and what researchers hope to achieve. Furthermore, my reading of the methodological / statistical literature leads me to believe that EFA is more likely to produce desirable outcomes.

I wonder whether the distinctions that you prefer to draw, although perhaps statistically sophisticated (I'm not in a position to assess that), might be somewhat idiosyncratic - i.e., not shared by other methodologists.

To save you and me going in circles, I'd welcome input from others.

Robert Trevethan: alright, then perhaps you could clarify by referring to a procedure from a known statistical package (of your choice).

Clearly, there is PCA in SPSS (which is most, and perhaps over-used), with CFA being available in the add-on AMOS. In R, there is for instance prcomp() for EFA and factanal() for CFA. If there was such a thing as a (separate) EFA, as you seem to imply, I would expect to find a dedicated statistical procedure implemented somewhere. But I just don't see it.

Adrian Dușa, I am accustomed to SPSS, where, under Dimension Reduction, PCA is the default option for Extraction. From what I have read, this leads many researchers to misleadingly assume to that PCA is a form of exploratory factor analysis - and I have read methodologists who criticize SPSS for perpetuating that misunderstanding.

Some light might be shed on this starting at the 3.50-minute mark in the following YouTube presentation: https://www.youtube.com/watch?v=f0ynIUyO9AU

So, yes, PCA is definitely available in SPSS - but so is EFA, with both orthogonal and oblique rotations available for EFA.

Thanks, but I'm already aware that SPSS cannot be used to perform CFA - and that the AMOS add-on should be used for that.

I confess to being even more confused by your answer. My question was: is there a dedicated menu in SPSS, specifically named Exploratory Factor Analysis?

As previously mentioned, EFA is only a broad term for many things, but not something completely separate from PCA. To clarify what David L Morgan might have referred to, one other data reduction method is principal axis factoring (most likely this is where the word "principal" comes from) but all of these are EFA. Orthogonal and oblique rotations are in EFA, they do not define EFA as separate from PCA.

The very video you've sent writes quite clearly, to use SPSS for either principal component analysis, or principal axis (aka) common factor analysis.

Adrian Dușa, sorry for not making myself clear. There is not a dedicated menu in SPSS that is explicitly labeled "exploratory factor analysis". One simply has to know what the options in SPSS mean - which, I acknowledge, could be a problem.

In the video I sent the URL for, my interpretation is that PCA is one form of extraction method within Dimension Reduction, and the other data extraction options are all within EFA (which unfortunately isn't stated) - and principal axis factoring is simply one of those extraction methods.

Unfortunately, it seems you and I are not reading the literature, or interpreting the options in SPSS, similarly.

Maybe someone else could contribute some ideas.

Robert Trevethan: with the risk of boring the other reads to death, could you please clarify if you consider PCA as something else (different) than EFA? What you just wrote (PCA being an extraction method, as opposed to all others which are within EFA) is highly ambiguous and suggest that PCA is not within EFA.

More specifically, what is your (own) definition of EFA?

My definition is clear: if one has a clear idea of the number of factors, and want to test if the data is an accurate representation of the underlying hypothetical (operational) model, that would be CFA.

Otherwise, if the number of factors is unknown, it is exploratory and PCA is without any doubt one such type of exploratory analysis.

Adrian Dușa, yes, there is a risk we could bore other RG members - but, in my view, sorting out the difference between PCA and EFA is worth pursuing. I wonder whether the attached document might help in that regard (it conforms with my view about the difference between PCA and EFA), but there are also a number of sites on the web that deal with the difference between PCA and EFA.

I agree with you that both procedures are exploratory, so I suspect the main issue that is confusing things between us is terminology . I would add that I think a lot of researchers use PCA when intending to engage in data reduction for three reasons: It is the default option in SPSS, they simply do what other researchers do (thus behaving like sheep), and they are unaware of the number of methodologists who generally discourage PCA and recommend EFA instead.

Adrian Dușa in case it helps, I've sketched out something that I hope illustrates where I'm coming from with regard to principal components analysis and (both types of) factor analysis. It's attached.

If anyone thinks that what I'm proposing is misguided or could be improved, I'd welcome the feedback.

This all strikes me as a tempest in a teapot. Following, SPSS these are all techniques for "dimension reduction," but PCA makes a distinctively different set of assumptions than the cluster of programs that Robert Trevethan is calling EFA. (The technical differences involve the assumptions around commonalities and uniquenesses.)

Principal Axis Factoring (which I mis-labelled Principal Factors) is one version of EFA, Maximum Likelihood factoring is another. What these EFA methods share is the same underlying model, but they use different statistical procedures to implement that model.

Given the difference in assumptions between PCA and EFA, I favor EFA for most routine attempts to create scales.

Finally, with regard to the debate about using ordinal variables (usually Likert-scored items) in either PCA or EFA, doing so does violate the underlying assumptions of these methods, but such a violation is a choice that literally thousands of published papers make every year (including papers that use AMOS or other such programs etc. for either Confirmatory Factor Analysis of Structural Equation Modelling).

David L Morgan, thanks for getting back with regard to the mis-spelling in your earlier post. Given the confusion that can result from terms such as principal components analysis and principal axis factoring, something that mistakenly lands midway between them, or sideways, can be confusing.

Like you, I prefer EFA to PCA, and I guess that's why you queried Peter Samuels about that up in an earlier post on this thread - where he recommended PCA.

I agree with you that there are "thousands of published papers" every year that are based on ordinal data but use procedures that assume interval data. To me, this raises the prospect of a tempest in a teacup. If useful results emerge in those published papers, might the strident arguments that Likert items can be regarded only as at the ordinal level ring a bit shallow?

I simply think that the conventionally argued delineation between ordinal versus interval data - and consequently nonparametric versus parametric statistics - might not always be useful or valid.

Robert Trevethan For me, the biggest issue is between using a single ordinal variable as if it were interval, versus combining multiple ordinal variables into a scale. For example, I personally would not use an ordinal variable as the dependent variable in a regression, but I would hesitate to use a high reliability scale as a DV.

Robert Trevethan and David L Morgan: thank you for your inputs, it does indeed seem to be a matter of definition and interpretation of published work. This exchange is very worthwhile, for those seeking to clarify very often confused terms.

David L Morgan, I agree with your most recent post right up until the last few words. Do you mean you "would hesitate" - or did you intend to write "would not hesitate"? That aside, I think that a lot of successful research being based on individual Likert items regarded at the interval, not ordinal, level of measurement should make us a bit cautious about declaring adamantly that single Likert items can only and always be regarded as ordinal in nature. The quotations I provided in an earlier post in this thread, from Hair et al. and Norman, provide support for that.

Adrian Dușa, thanks for your wrapping-up kind of comment. I hope you, indeed, did not regard our interchange as helpful - not confrontative. My aim was certainly to achieve some clarity and to help others. When you kept challenging me, I responded because I thought it would be rude if I simply ignored you. Every good wish!

"I would not hesitate to use a high reliability scale"

David L Morgan, thanks for this clarification / correction. I'd thought your original version was contradictory and might leave some readers perplexed.

On item level statistics, you may want to do or see how the responses perform on each item on the two types of scales then you may need a Wilcoxon Signed Rank Test or Wilcoxon Matched pair Test.