If you have performed exploratory factor analysis and mention that you have correlated factors, parallel analysis and weighted least squares are recommended. If you already have a previous factorial model you can perform the AFC directly.

Hello Ömer,

If I understand your query, you're trying to determine whether a previously-factored measure has the same structure with your sample. The logical starting place would be to first run a confirmatory factor analysis. If that tested structure does a poor job of reproducing the relationships observed in your data, and the misfit isn't something simple, then it's time to consider a different structure.

As Cristian suggests, an exploratory factor analysis would be logical follow-up.

Once you identified a structure that appears to work for your sample, and fits with the logic/rationale/theory underlying the measure, then your chore is to try to explain why (for your sample) the original structure doesn't hold.

The literature is full of examples wherein people administer the same sample to new/different samples, and find the apparent factor structure to be different from what has previously been published. Possible reasons include: (a) ordinary sampling variation; (b) use of different methods/decision-making (such as, how many factors to extract); (c) cultural/language differences; (d) a biased sample; or (e) some combination. It's not always easy to figure out what the truth is, though corroborating your new-found structure with additional data gives you confidence that you're on the right track (at least, for your target population).

Good luck with your work!

Dear David Morse and Cristian Ramos-Vera

Thank you very much for your answers.

As David Morse mentioned, there are a number of reasons for different results.

But would it be sufficient for exploratory factor analysis alone, with reference to these reasons?

Or as you mentioned, is it necessary to do both factor analysis together?

Best Regards..

Like David Morse, I am aware of examples of a factor structure not conforming to the structure that the original scale developers obtained - and I suspect for all the reasons that David has suggested above. So you're not on your own, Omer.

In some research that some colleagues and I have recently had published, the variations in the original scale's structure were so numerous within the literature, and we were surveying in a different culture with different "values" from previous samples, that we decided to start with exploratory factor analysis (EFA) because we felt we didn't have sufficient confidence in what the structure would look like to submit our data to confirmatory factor analysis (CFA).

One problem that we anticipated facing is that it is apparently not best practice to use both exploratory and confirmatory factor analyses on the same data set, so we thought that using only EFA would be best - particularly given our sample size of only around 400 with 24 items.

Since then, with a larger sample size of around 900 in a completely different project, we have conducted EFAs and CFAs on separate data sets - and felt comfortable doing so.

In essence, and I don't want to be argumentative, if you don't have a sufficient number of participants to run EFAs and CFAs with separate subsamples, I'd recommend going for EFA if you're not confident about the factor structure of your data.

I'm happy to be advised differently, but I think that EFA can provide more information than does CFA about which items might be good "performers" and how the items relate to each other within a particular data set. CFA, in my view, can return a pretty blunt "You've succeeded" or "You've bombed out". I'm also aware that the criteria for assessing the success of a CFA are not as clearcut as many might believe.

Dear Robert Trevethan

Thank you for your answer.

I assume that the reason why the statements did not spread to the factors as measured originally was the demographic difference and number of participants.

Because, within the scope of the research, I examined the attitudes of 191 high school students.

I cannot increase the number of participants as the research encompasses my entire universe.

I will consider your suggestions.

In both factor analysis, I try to determine a road map in case of similar results.

Best Regards...

You're really welcome, Omer - and sorry about not being able to get the umlaut for your name. I'm not surprised about your not being able to get more participants. That kind of thing is not atypical in real-life research.

I join David Morse in wishing you the best for your research. Working out the most accurate and honest way to analyse and report your data can be engaging.

Dear Robert Trevethan

I was really happy for your answer.

I also agree David Morse .

I'm looking for a reference to the criteria mentioned by David Morse (possible causes; a, b, c, d, e).

When I find a source related to this, I can provide a more accurate structure.

Best Regards...

Dear Omer,

David might well get back to you, but I suspect that his knowledge / wisdom comes from a fair amount of accumulated and invaluable experience.

Dear Robert Trevethan

I hope he returns.

I have come across examples of the criteria mentioned in the articles before. However, I am concerned that it is acceptable because it is not supported by any references.

Best Regards..

Hello Ömer, Robert, and anyone else following this thread,

The reasons for why my factor structure doesn't match your factor structure come from working with, and helping others work with data sets, and a lot of reading (of factor solution papers that disagreed with one another)! To my list, I will add one more possible reason: The measure under investigation is of questionable quality and the factor solutions differ because of measurement error and other noise in the data set, instead of the other possible reasons.

Feel free to quote me on this, lol! In reality, I don't think that your inability to find the same words in print prevents you from offering the reasons as possible explanations for structure discrepancies across samples.

Though I'm not volunteering to do so, I think it would be an interesting investigation to review the literature for how often structures apparently fail to be corroborated compared to how often they are confirmed in subsequent studies. I'd be willing to wager that the ratio is far greater than 1:1.

Good luck with your work.

Hello David and Ömer,

Thanks for getting back, David. Your responses are helpful and nourishing.

As it happens, some colleagues and I have recently had an article published in which the introduction contains exactly the kind of information you've thrown out the challenge about. It's an open-access article:

Ma, K., Trevethan, R., & Lu, S. (2019). Measuring teacher sense of efficacy: Insights and recommendations concerning scale design and data analysis from research with preservice and inservice teachers in China. Frontiers of Education in China, 14(4), 612–686

Because it's open access, I have already put an author copy on ResearchGate

Article Measuring Teacher Sense of Efficacy: Insights and Recommenda...

We looked at a range of publications in which preservice and inservice teachers had been administered the Teacher Sense of Efficacy Scale, and we found a range of outcomes within each of those two categories of participant. There were different numbers of factors in different publications, and different items loaded on different original factors, and sometimes the results were very messy.

I suspect that several reasons you've put forward, David, came into play.

Ömer, I've been thinking about it, and I think David is right: You don't need always need citations to be able to suggest reasons for your own results. Demonstrate you're an independent, thoughtful researcher! Go for it!

Following

Dear dr. Robert Trevethan

Very very useful contributions

Dear Dr. David Morse a

Very very useful contributions

Dear Dr. Cristian Ramos-Vera

Very deep analysis

https://www.sciencedirect.com/topics/agricultural-and-biological-sciences/discriminant-analysis