Hi Nadja,

first, when your are in a position to judge an item whether it loads on "its" factor versus "the other" factor, you seem to have some kind of measurement model in mind which makes the whole endeveour of the *exploratory* factor analysis obsolete. Hence, I would have started with a CFA in the first place. You could however still do it and incorporate external criteria of the factors in question---optimally variables that affect/or are being affected specifically by the factors.

Second, of course an item can be a valid measure of "the other" factor but given its question wording (which led you to believe that it is a measur of "its" factor, see above), the question arises about the meaning of the other factor when this item with its semantic content turns out to reflect this factor. Again incorporating validity criteria (see above) would help not only to (further) test the model structure (and reduces the problem a bit that the structure was explored) but also to evaluate whether the factors are what you think they are.

Perhaps you can be more specific and present the list of items, your interpretation of the factors and factor loadings.

Best,

Holger

Nadiah Farhanah Mohamad Fazil, my hunch is that it would be quite permissible to retain items that load, presumably unexpectedly, on a particular factor.

Perhaps the following would provide you with useful information about EFA:

Beavers, A. S., Lounsbury, J. W., Richards, J. K., Huck, S. W., Skolits, G. J., & Esquivel, S. L. (2013). Practical considerations for using exploratory factor analysis in educational research. Practical Assessment, Research, and Evaluation, 18(1), 6.

Briggs, S. R., & Cheek, J. M. (1986). The role of factor analysis in the development and evaluation of personality scales. Journal of Personality, 54(1), 106–148. https://doi.org/10.1111/j.1467-6494.1986.tb00391.x

Clark, L. A., & Watson, D. (1995). Constructing validity: Basic issues in objective scale development. Psychological Assessment, 7(3), 309–319. https://doi.org/10.1037/1040-3590.7.3.309

Costello, A. B., & Osborne, J. W. (2005). Best practices in exploratory factor analysis: Four recommendations for getting the most from your analysis. Practical Assessment, Research & Evaluation, 10, 1–9.

de Winter, J., Dodou, D., & Wieringa, P. A. (2009). Exploratory factor analysis with small sample sizes. Multivariate Behavioral Research, 44(2),147–181. https://doi.org/10.1080/00273170902794206

Fabrigar, L. R., Wegener, D. T., MacCallum, R., C., & Strahan, E. J. (1999). Evaluating the use of exploratory factor analysis in psychological research. Psychological Methods, 4(3), 272—299. https://doi.org/10.1037/1082-989X.4.3.272

Gaskin, C. J., & Happell, B. (2014). On exploratory factor analysis: A review of recent evidence, an assessment of current practice, and recommendations for future use. International Journal of Nursing Studies, 51(3), 511–521. https://doi.org/10.1016/j.ijnurstu.2013.10.005

Hair, J. F., Jr., Black, W. C., Babin, B. J., & Anderson, R. E. (2014). Exploratory factor analysis. In J. F. Hair Jr., W. C. Black, B. J. Babin, & R. E. Anderson (Eds.), Multivariate data analysis (7th ed., pp. 89–149). Pearson.

Hayton, J. C., Allen, D. G., & Scarpello, V. (2004). Factor retention decisions in exploratory factor analysis: A tutorial on parallel analysis. Organizational Research Methods, 7(2), 191–205. https://doi.org/10.1177/1094428104263675

Howard, M. (2016). A review of exploratory factor analysis (EFA) decisions and overview of current practices: What we are doing and how can we improve? International Journal of Human-Computer Interaction, 32(1), 51–62. http://dx.doi.org/10.1080/10447318.2015.1087664

Matsunaga, M. (2010). How to factor-analyze your data right: Do’s, don’ts, and how-to’s. International Journal of Psychological Research, 3(1), 97–110. https://doi.org/10.21500/20112084.854

Preacher, K. J., & MacCallum, R. C. (2003). Repairing Tom Swift’s electric factor analysis machine. Understanding Statistics, 2(1), 13–43. https://doi.org/10.1207/S15328031US0201_02

Russell, D. W. (2002). In search of underlying dimensions: The use (and abuse) of factor analysis in Personality and Social Psychology Bulletin. Personality and Social Psychology Bulletin, 28(12), 1629–1646. https://doi.org/10.1177/014616702237645

Tabachnick, B. G., & Fidell, L. S. (2007). Using multivariate statistics (5th ed.). Pearson Allyn & Bacon.

Zwick, W. R., & Velicer, W. F. (1986). Comparison of five rules for determining the number of components to retain. Psychological Bulletin, 99(3), 432–442. https://doi.org/10.1037/0033-2909.99.3.432

It sounds like you are waiting bits of both exploratory and confirmatory factor analysis. Essentially, you want to allow item influence to leak onto factors other than the one that you expect to be mostly affected by. The most discussed procedure for this is exploratory structural equation modeling. An example reference is: https://www.vanderbilt.edu/psychological_sciences/graduate/programs/quantitative-methods/quantitative-content/marsh_morin_parker_kaur_2014.pdf