I would almost always recommend correlated factors (i.e., oblique rotation) for social science data. Next, I think you need to examine the correlation matrix among the factors. How is the factor in question related to the other factors? In particular, what is the chance that is it negatively correlated with all the other factors?

Martin Brygger Andersen - hello again, Martin - I have some suggestions.

1. As David Morgan has recommended, oblique rotations are almost always preferable (despite many, or most, researchers using varimax rotations - possibly because they just do what everyone else is doing).

2. I'm not sure how you determined the number of factors in your data but, if you used a varimax rotation, I suspect you used the Kaiser criterion (eigenvalues > 1) because that's what most people do. However, that method has been disapproved of for about 30 years by good methodologists because it is likely to suggest there are more factors in your data than are really there. I recommend you use the scree test or, better, parallel analysis. The latter is very easy to perform. Ask me for details if you want.

3. Your problem with all items loading negatively might be quite "superficial" and easily solved. The same thing happened with us recently when we used oblimin as our rotation. Using promax (another oblique rotation) made the problem disappear as if by magic.

Let us know how you go!

4. I have some concerns if you're using the same data to perform your EFA(s) and CFA(s), but that's a different kettle of fish.

Hello Martin,

In addition to the good advice you've already received, I'd add a simple note that many rotational schemes are utterly arbitrary when it comes to the issue of whether the majority of variable loadings are positive for any given factor. Since a 180-degree reflection of a factor doesn't change it's structure (e.g., how the individual variables do or don't ally with that factor), but instead just reverses the signs, it's one option to multiply all associated values by -1.

However, David Morgan's reply urges you to think about the theory (or whatever rationale you have for the respective factors) as to whether the factor in question and its affiliated variables should correlate in a specific direction (positive or negative) with other factors. That judgment should over-ride any question about whether reversing signs for variable loadings.

Good luck with your work.

Thanks for the great answers,! I really appreciate that!

1. First, I looked at the correlation matrix, as David Morgan suggested, and all correlations were positive. It does not make theoretical sense with negatively correlated factors. I believe that the results are caused by some degree of common methods bias or strong correlation between two factors in the model. Thus, I have taken that into account when interpretating the data.

2. Using Promax (oblique rotation) with principal axis factoring, the problem was seemingly solved as all factor loadings are now positive (thanks to Robert Trevethan). 50.6% of the the variance is explained. Through deleting items with the lowest factor loadings (which also created some problems in the CFA), the amount of variance explained is improved to 53% (I wonder whether this is sufficient?).

3. I used a combination of interpretating eigenvalues and the scree plot and extracted 4 factors. There are definitely 3 factors in the data but two of the factors are quite strongly related (correlation of .55). However, Gaskin (2020) suggests that discriminant validity is sufficient as long as there is no inter-factor correlation above .7. To be on the safe side, I interpret the parameter estimates cautiously between the two factors in the CFA.

The contributions by David Morgan and David Morse have reminded me of three things that I should probably have mentioned above.

The first is the importance of relying on theory and interpretability when considering the outcomes of EFA - and that is what we did when confronted with a complete factor with negative item loadings: The set of negative loadings was completely "illogical" in relation both to theory and to the contents of what was the only other factor.

The second is that I think there is a problem if only some items within a factor have negative loadings - which is different from all items having negative loadings.

The third is that, when we were confronted with our problem, two things happened. First, a helpful person I'd recently met on RG suggested moving from oblimin to promax as the rotation. I followed that up by doing a search on the web related to our problem and found a post from someone who had been good enough to place a post about her having had the same problem, and that she solved it by moving from oblimin to promax for her rotation. My main point here, I guess, is that RG can definitely be an excellent platform for obtaining and sharing information.

Hello Robert Trevethan,

Thanks for your suggestions.

I am using the same data on the EFA and the CFA. I believe you would recommend a split-data approach rather than using the same data?

Martin Brygger Andersen, thanks for coming back with your feedback. I'm really pleased that using a promax rotation solved the "sign" problem. I should perhaps have mentioned that using promax rather than oblimin also resulted in a slightly more satisfactory solution for us, with two items no longer cross-loading. (Please don't ask me why promax sometimes produces a more desirable solution than does oblimin; it is just so . . . )

It's also interesting that, as I'd suggested, the Kaiser criterion can often suggest more factors in a dataset than are really there - in your case that there are three, not four factors. I dare you to use parallel analysis, however, because the methodologists I respect recommend it. Parallel analysis used to be difficult, so researchers swerved it. However, to demonstrate how easy it can now be done, I'm attaching an example from some of our own research using a readily available website that works beautifully.

The other thing, Martin, is that you've not indicated whether you're using the same data for your CFA(s) as you did for your EFA. If so, please be aware that doing so isn't really approved of because you're essentially going in a self-predictive circle. You should really use a fresh sample for the CFA(s).

Well, that issue is easily solved as I have data from 2015, 2017, and 2019. I can simply use the EFA on data from an earlier sample. I can cross-check the results on the other samples, to check whether the same factor solution arises?

Thanks for the file, I will take a look and download it to my library!

Martin Brygger Andersen, I'm pleased for you (and that I raised the issue about the need for different samples for EFAs and CFAs).

Here's something else that might be of interest in relation to your CFAs: Recently, a colleague and I have done some CFAs in which we chose different numbers of items to analyse (10 versus 5 items on each factor) and the items restricted to those that had either high or low loadings in the other sample's EFA.

We found that the fewer the number of items, and the higher the loadings on those items, the more we were able to obtain desirable goodness-of-fit (GoF) outcomes.

That led me to do some exploring, and I've found that, over the last 10 or so years, good methodologists are demonstrating that the conventional rules-of-thumb for GoF indices (e.g., that the RMSEA should be < .05) have some quite questionable foundations.

That might not trouble you, but certainly in our case we could have aimed to satisfy conventional GoF indices at the cost of having content validity (wide construct coverage) in our scale.

Hello Trevethan,

I had the same experience in relation to the article I am writing now. In the first edition, I used CFA on 28 variables. By reducing this to 22 variables (and removing items with lower factor loadings rather than allowing error terms to correlate), the fit indices were overall improved in the model.

The RMSEA is close to .04 and and GFI is .97 and CFI is .96. I am paying much more attention to model fit and I would rather sacrifice a few items to improve the overall fit of the model. I had to remove some items though that reduced the coverage of the measured concepts (e.g., an item on bullying). Furthermore, the average variance extracted (AVE) on each factor is much higher when using fewer items. This is especially important when using reflective indicators to my knowledge.