I don't see why you wouldn't want to examine all four of your measures. After all, you want to be certain that each of them performed adequately in your sample, rather than simply that this was the case.

Thanks David L Morgan !

Actually I did the factor analysis for the four measures, but the result was somehow disappointing :(

Some items of the international measures didn't load on their respective factors. Moreover, the explanatory factor analysis failed to reproduce the same number of dimensions in each measure. We can remove some items in order to reach the desired dimensionality, but in this case, we lose the chance to compare our results with literature and with our future samples. We intend to repeat this study on samples of different countries. So, the question remains: Is it "mandatory" to conduct factor analysis for all items in the questionnaire?

If your factor analysis results aren't the same as the published results for the international measures, then you have already lost the ability to compare the results due to those differences -- regardless of whether you drop the items. So, you have a choice between accepting the full results as the best representation of how your sample responds to these items, or dropping items to match your original expectations.

One way to address this question would be through Confirmatory Factor Analysis, which could allow you to test the difference between your observed and expected models.

Thanks again David L Morgan

We already have two samples from different countries which answered the same questionnaire. I run Confirmatory Factor Analysis for each sample separately and many items were dropped in order to reach the desired factors. However, results of CFA from both samples were different with regard to those self-developed inventories. I got different factor structures with different items in each factor!

Now, how can I run further analyses to compare between those two samples while having different factor structure?

I saw some comparative articles ( for example: Article Teacher efficacy: A comparative study of Hong Kong and Shang...

I am still confused about what is the best way in the analysis for a comparative study. Any suggestion/ recommendation would be appreciated!

Tahani Z. Aldahdouh, I have just noticed your question(s), and I might be able to help if you are doing research in the area of teacher sense of efficacy.

Some colleagues and I are doing research in that area, and part of that has involved us inspecting the factor structure of the Teacher Sense of Efficacy Scale. Although many researchers seem to accept on face value that it has 3 distinct factors, we have brought forward evidence from a number of studies that demonstrates the factor structure to be highly unstable.

Not only are there different numbers of factors depending on the samples, but items move from one factor to another - again depending on the samples. The study you mention about Hong Kong and Shanghai teachers (I think I'm familiar with that one: Cheung, 2008?) is quite nice, but it demonstrates the inconsistencies to only a small extent. The inconsistencies are a lot more troubling in other research - as, indeed are the statistical practices and conclusions of many researchers in the area.

I hope that's helpful.

If you have to keep dropping items to reach a fit with the "desired" model, then this strong evidence that the structure of the relationships among the responses in your current sample(s) does not match the externally generated model. And if you have to drop different items in your two different sub-populations, this is further evidence that external model does not match the pattern of responses in your current sample(s).

In general, the process of dropping items is a more acceptable practice in EFA, where you want create factors that are limited to items with strong loadings. In contrast, for CFA the goal is to test models -- and to reject models that do not fit the data.

David L Morgan, thanks for contributing to this thread. I'd like to discuss something with you, however - something that has been troubling me for some time. It's the business of retaining items with strong loadings.

My sense is that, if that is done, there is a risk of ending up with factors each of which has limited domain coverage. Instead, if one retains items that have a range of loadings, as long as all of those items have, say, loadings above .40 and do not have cross loadings with other factors, breadth of domain coverage will be more satisfactory.

I have seen scales in which only high-loading items have been retained, and my sense is that those items have a high degree of redundancy among them - both superficially (just eye-balling them) and as indicated by correlations.

What do you think, please? I'd appreciate your perceptions.

Tahani Z. Aldahdouh, I recently come across a similar situation, where construct structure in two different groups(samples) was different, if I run the model separately. I wanted to compare the samples which was kind of meaningless in given situation, as construct structure was difference.

After discussions and deliberations, I opted to run the model on combined data set to examine the outer model/measurement model. In this way, then ran a multi-group analysis. While doing so I also ran a measurement in-variance test for multi-group, to ascertain differences in construct's structure. It solved my issue.

I am using structural equation modeling (SEM) in SmartPLS environment and you are probably using SPSS envionment.

Also, have a look at following discussion: https://www.researchgate.net/post/how_to_compare_multi-item_constructs_between_2_groups

Thank you Robert Trevethan for your contribution and sharing your experience relating to the inconsistency of teacher efficacy beliefs scale.

Actually we used efficacy beliefs' scale different than that mentioned in Cheung (2008). We adapted items from Dellinger’s et al. (2008) scale of Teacher Efficacy Beliefs System-Self (TEBS-Self). However, we got one factor structure while the original scale comprises 3 factors!

Thank you Ali Farooq for sharing your experience with this same situation! I read the discussion you posed in your provided link and there the answers were really helpful and insightful!

Thank you once again!

Thank you David L Morgan for this clarification.

Tahani Z. Aldahdouh, good to hear from you. Interestingly, we also obtained only a single factor on both occasions when we administered a slightly adapted version of the Teacher Sense of Efficacy Scale (TSES) to groups of about 450 Chinese preservice teachers and about 350 Chinese inservice teachers. We certainly didn't get the 3 factors that many people think are in the TSES. There could be several reasons for that (we discuss these in a journal article that is currently under review) - including that others tend to use varimax rotations which are likely to produce separate factors even when the factors are quite highly correlated. Cultural differences might also play a role

If you want to discuss our research at all, more privately, please feel free to give me your email address. Alternatively, we might be able to provide greater benefit to others by continuing our discussion here.

I confess that I haven't heard of the scale you used. I must try to look it up.

Robt.

Hi Robert Trevethan, sorry for my late response.

Yes, it would be useful for interested researchers to discuss the differences in dimensionality of efficacy beliefs scale.

TEBS-Self scale(Article Measuring teachers’ self-efficacy beliefs: Development and u...

Tahani Z. Aldahdouh, no problem about your delay in responding. Thank you for mentioning the TEBS to me. (I don't know why I hadn't heard of it before.) I obtained a copy of the 2008 publication, and it came in handy because we had just done some research as the first stage of developing a new TSE scale, and some of the items on the TEBS gave me ideas for improving some of our items.

Incidentally, I'm glad you used PAF extraction and an oblique rotation. We used PAF and oblimin rotation - but the rotation was still oblique.

Ha ha, about you reading about our research once published. We have had some difficulty getting past reviewers, I think because we have been critical of quite a lot of research in the past, some of which the reviewers themselves might have produced. Reviewers can certainly be obstructive and self-protective gatekeepers. But, not to be defeated, we currently have 3 articles under review and two more in preparation.

All the best with your research.

If the question is whether the items measure multiple different factors or a single underlying factor, then the best approach would be to use Confirmatory Factor Analysis, which can explicitly test those assumptions.