Deciding on which items to remove in Exploratory Factor Analysis (EFA) can be a crucial step in refining your measurement instrument. While there are no hard and fast rules, here are some general guidelines to consider when removing items in EFA:

1. Cross-loadings: Cross-loadings occur when an item shows a substantial correlation with more than one factor. If an item consistently shows cross-loadings across multiple iterations of the analysis, it suggests that the item may not be a clear indicator of any single underlying factor. In such cases, it is advisable to consider removing or revising the item to improve the interpretability of the factors.

2. Significance: Significance tests in EFA assess the statistical significance of factor loadings. Non-significant loadings indicate weak relationships between items and factors. However, it's important to note that statistical significance alone is not always the sole criterion for removing an item. Substantive and theoretical considerations should also be taken into account. If an item is theoretically meaningful and substantively important, even if it has a non-significant loading, you may choose to retain it.

3. Theoretical relevance: Consider the conceptual meaning of the items and their theoretical relevance to the construct you are measuring. Items that are conceptually related to the construct and align with your proposed model should generally be retained. Removing items solely based on statistical criteria without considering their theoretical relevance may result in a less valid measurement instrument.

4. Iterative process: EFA is often an iterative process where items are evaluated and refined based on statistical analyses and theoretical considerations. It may be helpful to conduct multiple iterations of the analysis, removing items gradually and reassessing the factor structure after each removal. This iterative process allows you to evaluate the impact of item removal on the overall factor structure and interpretability of the factors.

5. Expert judgment: In addition to statistical criteria, expert judgment and domain knowledge are crucial in item selection. Seek input from subject-matter experts or colleagues familiar with the construct you are studying. Their insights can help validate your decisions regarding item removal and ensure that the final measurement instrument is theoretically sound.

Remember that EFA is a data-driven technique, but interpretation and judgment are necessary to make informed decisions about item removal. The goal is to arrive at a factor structure that aligns with your theoretical framework, has clear and interpretable factors, and reflects the construct you aim to measure.

Before deleting any item, you should evaluate if your model fits the data well. Are you estimating your model using Maximum Likelihood?

Thanks Esteban Montenegro

I am using principal axis factoring.

Hi Alireza Falakdin

Thanks for sharing your knowledge with me.

I am concerned that if I can run EFA a few times. After removing non-significant items and cross-loadings and rerunning the EFA, the result showed new non-significant items and cross-loadings.

So, I need to run around 6times to get my final EFA result.

My supervisors said that I should run EFA twice to get the final result. Running it a few times seems like manipulating the result.

I tried to find some references or guidebooks in this area but didn't find any.

Best wishes

Viola

Dear Viola,

I understand your concern about running EFA multiple times and the perception of manipulating the results. It's important to ensure the integrity and transparency of your analysis. While it is true that running EFA multiple times and making decisions based on the results could potentially introduce bias, it doesn't necessarily mean that it's inappropriate or manipulative. Exploratory Factor Analysis is an iterative process that involves refining the measurement instrument based on statistical analyses and theoretical considerations. It is common to run EFA multiple times to evaluate the stability and consistency of the factor structure and to refine the item pool.

However, to address your concern and maintain rigor in your analysis, I would suggest the following:

1. Pre-specify your analysis plan: Before conducting the EFA, clearly outline your analysis plan, including the criteria for item removal. Specify the criteria based on statistical significance, cross-loadings, and theoretical relevance. This helps establish transparency and ensures that you are not making ad hoc decisions.

2. Justify your decisions: Document and justify each step of the item removal process. Explain why certain items were removed or retained based on statistical results, theoretical considerations, and expert judgment. This helps demonstrate that your decisions were grounded in sound reasoning and not arbitrary.

3. Seek expert input: Engage with your supervisors or other experts in the field to discuss and validate your decisions. Their input can provide additional insights and help ensure the robustness of your analysis.

4. Consider alternative analysis techniques: If you're concerned about the subjectivity of item removal in EFA, you might consider exploring alternative techniques such as confirmatory factor analysis (CFA) or item response theory (IRT). These techniques allow for more explicit testing of specific hypotheses and can provide a more confirmatory approach to assessing the factor structure.

Lastly, while it is understandable that you couldn't find specific guidance in this area, it is common for the decision-making process in EFA to be iterative and involve multiple iterations. However, it's essential to maintain transparency, rigor, and adherence to your pre-specified analysis plan to ensure the validity of your findings.

I wish you the best of luck with your analysis, and I hope this information helps you address your concerns effectively.

Best regards,

Alireza

I would evaluate the model using maximum likelihood estimation. I would guide my decision based on model fit instead of p-values plus other sources of information such as parallel analysis.

Thanks a lot! Alireza Falakdin Esteban Montenegro

Please review the attached

Thank you！ @Afshan Hafiz