Hello Jessica,

Factor solutions can include variables that show salient affiliation with more than one factor (some personality measures are notorious for this type of structure). However, the concerns associated with cross-loading are usually: (a) the structure is more complex than Thurstone's idealized concept of "simple structure"; (b) it may make the task of characterizing what a factor represents more challenging; and (c) perhaps the variable isn't as well measured or defined as it could have been.

The answer to your question depends on your specific research aims as concerns this set of variables (and how these do or don't reflect the concerns listed above). If your goal is to derive the "cleanest" possible structure, then throwing out variables/items may be the way to go. Do recognize that the possible concern here is that you end up defining very restricted factors that may not fully represent the target constructs. As well, depending on your sample, it's possible that the resultant structure to choose may be overfitted to the sample and not generalize as well to other data sets.

In any event, if you elect to retain cross-loading items, then:

1. Yes, they are included in any representation of a factor (e.g., an estimated factor score, or just a summated score), or for score reliability estimates;

2. You present a factor structure matrix/table that shows all variables that you deem salient with each factor (factor pattern and factor inter-correlation matrices as well, if you used an oblique rotation);

3. If all loadings on a factor are negative, then you may reverse the sign and characterize the resultant variate as a reversed polarity construct estimate. (You may do the same with mixed sign loadings, as long as you reverse each variable's sign accordingly.)

4. The facetious answer is, if you get down to two items or fewer, you've likely gone too far! The more serious answer is, there is no way to predict in advance how many "keepers" there are from a preliminary set of variables that were constructed, identified, or adapted to tap one or more constructs. That's why people engage in EFA or CFA in the first place; to help identify what structures are supported by data and what structures are not.

The final note I would make is that Cronbach's alpha for an "overall" scale score might not be the best indicator, especially when you have identified multiple factors for that batch of variables. For individual factor scales, sure.

Good luck with your work.

Hello Jessica,

Factor solutions can include variables that show salient affiliation with more than one factor (some personality measures are notorious for this type of structure). However, the concerns associated with cross-loading are usually: (a) the structure is more complex than Thurstone's idealized concept of "simple structure"; (b) it may make the task of characterizing what a factor represents more challenging; and (c) perhaps the variable isn't as well measured or defined as it could have been.

The answer to your question depends on your specific research aims as concerns this set of variables (and how these do or don't reflect the concerns listed above). If your goal is to derive the "cleanest" possible structure, then throwing out variables/items may be the way to go. Do recognize that the possible concern here is that you end up defining very restricted factors that may not fully represent the target constructs. As well, depending on your sample, it's possible that the resultant structure to choose may be overfitted to the sample and not generalize as well to other data sets.

In any event, if you elect to retain cross-loading items, then:

1. Yes, they are included in any representation of a factor (e.g., an estimated factor score, or just a summated score), or for score reliability estimates;

2. You present a factor structure matrix/table that shows all variables that you deem salient with each factor (factor pattern and factor inter-correlation matrices as well, if you used an oblique rotation);

3. If all loadings on a factor are negative, then you may reverse the sign and characterize the resultant variate as a reversed polarity construct estimate. (You may do the same with mixed sign loadings, as long as you reverse each variable's sign accordingly.)

4. The facetious answer is, if you get down to two items or fewer, you've likely gone too far! The more serious answer is, there is no way to predict in advance how many "keepers" there are from a preliminary set of variables that were constructed, identified, or adapted to tap one or more constructs. That's why people engage in EFA or CFA in the first place; to help identify what structures are supported by data and what structures are not.

The final note I would make is that Cronbach's alpha for an "overall" scale score might not be the best indicator, especially when you have identified multiple factors for that batch of variables. For individual factor scales, sure.

Good luck with your work.

It will also depend on the type of estimation and rotation of the EFA.

Jessica Morgan, David Morse has given you some good advice up above, but I'd like to focus on some quite basic things that newcomers to factor analysis often don't attend to sufficiently. So, I'll ask:

Did you have enough participants? With 37 items, a rule of thumb would be that you need around 350 participants for factor analysis to "run" effectively.

How did you determine that you had five factors? Did you, for example, conduct parallel analysis (not difficult, as it happens, though few researchers seem to do it), but, instead rely on the number of eigenvalues greater than 1 - which the experts criticise as likely to indicate too many factors.

In what way did you rotate your factors? If you used an orthogonal rotation, that might have created problems.

Incidentally, with 27 items, a coefficient alpha value up above .90 is almost inevitable, even with a lot of items that don't correlate very well with each other. Alpha is very much dependent on the number of items thrown into the pot.

Thank you all for your help and advice.

Robert Trevethan

Just to answer a few of your questions:

• I had 213 participants (using the rule of 5 participants per item and 10% for errors).
• I determined 5 factors by initially using eigenvalues greater than 1, which presented 10 factors. However, I then used the scree plot to determine the change of direction (elbow) alongside the percentage of cumulative variance to determine 5 factors (51.95% of the cumulative variance accounted for).
• The initial solution didn't make any theoretical sense, so I used an oblique rotation since 3 of the factors seemed to correlate with one another.

David Morse

The aim of my study was to create a scale which measures a person's possession of power. I based my definition of power and the items of the scale on 7 constructs which would contribute to a person's possession of power.

I was not setting out to do a CFA, because the aim was to create a reliable and valid scale which measures this construct effectively. However, I was curious to see whether the factors reflected the sub-constructs that I based my items on.

Sure enough, the factors are fairly similar to sub-constructs. The sub-constructs are likely to be related to one another, which I believe is a possible reason for the cross-loading, as well as having some very vague/general items which could fit into any of the factors.

At this point, I am considering accepting the scale as a whole and rejecting items which bring the Cronbach's alpha down, and accepting that I may not have enough participants to determine well-defined factors. Furthermore, I'm not sure, at this point, if the study would be generalisable since many of my participants were white women aged 30-40 from the UK, and so I am reluctant to restrict the factors to meet this demographic, possibly meaning that it is overfitted to the sample and not measuring the target construct, as you explained David Morse.

Please let me know your thoughts and thank you again for your help. It is very much appreciated.

Jessica

Jessica Morgan, not all. My first impression is that you might have too few participants to feel confident about the results that factor analysis is producing for you. Factor analysis is one of those techniques that often needs biggish numbers before it works properly. There are times when smaller numbers, such as you've used, will be OK, but I think it's risky.

There are certain conditions that make smaller samples permissible, e.g., if communalities are high, but it can be a bit complex determining what's what WRT that.

Apart from that, might I encourage you to use parallel analysis to work out how many factors you have in your data. The scree plot is often good (and much better than the Kaiser criterion based on eigenvalues > 1). I am attaching an example of parallel analysis from the data of a fictitious person called John. You just need to start with the eigenvalues from your own data, go to the website indicated, and type in your number of participants and number of variables, then make the comparison in the same way I've done in the example.

Also, if I were you, I'd not remove all of the low-loading and cross-loading items in one batch. I'd remove them one, or a few, at a time to see what difference their absence makes in the factor structure of your data.

I think that's all I can suggest at this stage.

Thank you Robert Trevethan.

I will conduct the parallel analysis and see how I get on. I am now also concerned about the sample, not having enough participants and also not having a very diverse sample.

I have tried removing the cross-loading items in many different ways, and I have re-run the rotated FA a number of times with various combinations of items.

I have found a simple solution with no cross-loading items. This solution would have a total of 23 items (14 removed) with no cross-loading. This also has 5 factors with between 3-7 items per factor.

However, this does take the scale's Cronbach alpha below .90 (.881), which was to be expected, and my factors' alphas range from .67-.805. I am yet to conduct my validity checks, I'll be doing that this afternoon.

I think what I will conclude is that there does seem to be an underlying structure of the scale, but unfortunately I do not have enough participants to confidently conclude what it is.

Thank you!

Jess

Jessica Morgan, pls do have a go at the parallel analysis. Get back if you have problems. It looks a bit spooky, but it's really quite simple if you just sit down and do it! If it tells you that you have three factors, that's what I think you should aim for.