Hey Chelly,

usually you would not delete an item in an EFA design. Because if you'd do so, you'd assume an underlying factor structure and be more or less in a CFA design.

One of the best books for applied CFA is from Brown: Confirmatory Factor Analysis for Applied Research. There you have a chapter devoted to the problem you are describing. For the EFA, I can recommend "Exploratory Factor Analysis" from Holmes Finch.

Best wishes,

Sebastian

Sebastian Kurten hi sebastian,

thanks for your answer. I'm a bit confused because I'm following the recommendations of most scale development articles in which items in the EFA are deleted if it (a) had a communality loading of less than .4, (b) had an item-factor loading lower than .5 on a primary factor, (c) had high inter-item correlations as indicated by the anti-image correlation matrix (Tabachnick & Fidell, 2007). the goal of the EFA is to explore the factor structure, rather than confirming a predefined structure, so naturally there will be items which don't load on any factor and, therefore, do not reflect a construct/indicator

Chelly Maes, from what you have written, I think you are on the right track. More specifically, it seems that you don't want/need to use confirmatory factor analysis yet, and you have accessed some relevant literature.

Maybe that literature doesn't "hold your hand" enough, however. Here are some things I'd do:

I'd not be quite as severe as you initially. Some items with (extracted) communalities that are < .40 can end up looking better when other "defective" items are removed. The same goes for items with loadings < .50.

So, I'd begin by removing items with really low extracted communalities (down around 1.5), items with low loadings (say, < .30), and single items that load on only one factor, but maybe not items that have cross-loadings < .20 - and see what happens. I'd keep an eye on whether the number of factors changes as a result of that. And I'd keep doing that kind of thing, and a little bit more, VERY CAUTIOUSLY until it seems you have some clear factors, each with no cross-loading items and at least four items.

I'd add that I think you should have used parallel analysis (certainly not the Kaiser criterion) at the start to determine the likely number of factors in your data, AND that you use an oblique rotation (e.g., oblimin or promax) rather than an orthogonal rotation (e.g., varimax).

Offhand, I can't think of an article or book that supports the above. It just seems to be what those in the know engage in.

Every good wish for what you're researching.

Robert Trevethan Thank you!

Chelly Maes, you're really welcome. Feel free to get back with queries, etc. I think that exchanging information publicly, as in RG threads, is good in that others can be exposed to ins and outs relevant to them, but if you feel shy about any aspects of your work, feel free to contact me privately through RG.

Robt.

I have used the steps mentioned by Robert in one of my papers. I usually don't suggest my own work in replies, however, I believe this short paper will reinforce your approach - that you are on the right track:

Farooq, A., Alifov, S., Virtanen, S., & Isoaho, J. (2018, July). Towards comprehensive information security awareness: a systematic classification of concerns among university students. In Proceedings of the 32nd International BCS Human Computer Interaction Conference 32 (pp. 1-6).

The following paper is one good guide for EFA:

Costello, A. B., & Osborne, J. (2005). Best practices in exploratory factor analysis: Four recommendations for getting the most from your analysis. Practical assessment, research, and evaluation, 10(1), 7.

Robert Trevethan Thank you, that's very kind! I will!

Ali Farooq oh that's what I was looking for, thank you very much!!

Ali Farooq, it was generous of you to share your paper with Chelly. I have read it with interest, and I noticed that you had 354 participants and 74 items. Because one of the common recommendations is that there be 10 participants for each item, I wonder what results you might have obtained if you had more (around 750) participants. Of course, there are different recommendations concerning the ratio of participants to items, so your results might not have been different.

I also notice that you worked with 11 factors as a result of using the Kaiser criterion. I wonder whether you'd have decided to work with fewer factors if you had used parallel analysis to identify the likely number of factors in your data. (Parallel analysis is now very easy to perform, and it has been recommended as preferable to the Kaiser criterion for about 20 years, so I'm not sure why so few researchers use it. Incidentally, things with regard to ease of conducting parallel analysis have improved a lot since Costello and Osborne's very helpful article was published 15 years ago.)

I confess that I'm not sure how you and your colleagues moved from having 11, to having six, factors.

Anyway, I must say that I'm impressed with the % of variance your factors accounted for - though maybe that's because of the number of factors that you extracted.

I'd LOVE to have played with you in analysing your data! Your research looks fascinating.

Chelly Maes, it might be helpful if I indicate that, in my experience, data have usually shaken themselves into "respectability" after only one or two iterations. However, some data a colleague and I are currently working with have needed about eight attempts to knock into shape. I think that's because there was extremely little variability in the data because the items weren't particularly appropriate for the people being surveyed.

Unfortunately, I don't have fully written-up and published accounts of research demonstrating the kind of thing you might like to see.

Do feel free to add more to this thread or to contact me privately.

For the moment, it might be helpful if I attach some information about conducting parallel analysis. That procedure might seem dauntingly complicated at first, but it really is extremely easy - and I have found it much better than either the Kaiser criterion or the scree test for identifying the likely number of factors in data.

Robert Trevethan I totally agree that ideally, we should have 10 times the number of items for the analysis. However, in practice, finding 750+ responses is a headache especially when are 80+ items (including demographics). I tried but I couldn't find more participants. This is one reason that the study ended up as a short paper rather than going to an impact factor journal. :D

And you are absolutely right that parallel analysis is better than Kaiser's criterion. I learned about it after this paper and used it afterward. It is super easy to run. Actually, I recommend everyone, who seeks help, to use parallel analysis.

We went with a subjective coupling (qualitatively) of factors to areas (11 to 6). I am not aware of any other way to do and 2nd level coupling/grouping. This grouping was necessary for my doctoral research.

I must say, I have learned a lot from RG, from experienced people like you, and I still learn many new things by following your comments.

Chelly Maes Here is another paper that will definitely be helpful for you; please check paragraph 3 on the first page:

Beavers, A. S., Lounsbury, J. W., Richards, J. K., Huck, S. W., Skolits, G. J., & Esquivel, S. L. (2013). Practical considerations for using exploratory factor analysis in educational research. Practical Assessment, Research, and Evaluation, 18(1), 6.

Ali Farooq, it is good of you to have suggested the article you've cited in your post immediately above here. The first sentence of the third paragraph on p. 1 is particularly relevant, and I wonder whether the final paragraph on page 11 of that article might also be helpful for Chelly's purposes. That's the paragraph that, at the start, mentions factor analytic processes being "iterative". It's only a brief mention of the issue that Chelly wanted information about, but it might help to serve the purpose for her.

Apart from that, thank you for your post two up from here. I'm glad that my posts have been helpful - and I fully appreciate that it's sometimes difficult to conduct the kind of research that we'd ideally like to conduct. I'm also glad that you endorse parallel analysis for identifying the number of factors in data AND that you provided reassurance about its being easy to do.

All the best with your own research.

Thank you, Robert Trevethan . I have learned a lot about data analysis techniques from the discussions in RG, especially when experienced persons such as you jump in and guide. Keep us inspired with your guidance and feedback! :)

Robert Trevethan and Ali Farooq thank you for the additional sources! This really helps :)