I recently collected data which has subsequently been analysed using PLS-SEM, however when I try to perform additional analyses as part of the empirical component of a related article some items from the BFI do not load. For example, the pattern matrix in SPSS used to test validity before conducting a CB-SEM analysis shows items loading on other personality constructs. I have some items of Extraversion loading on Agreeableness etc.
Data was collected using an English instrument in the US, and since the PLS results make sense I find it strange that my CB-SEM (now using AMOS) approach now causes these issues. Is my analysis even valid if I eliminate some of these items? Apologies for the lengthy start to this discussion.
I think you should concentrate on using AMOS to evaluate your measurement model. In particular, you can specify the model according to the original theory and then examine the modification indices. If necessary, you may need to include correlated errors to indicate "cross-loadings."
Timo Mandler : Thanks so much for this feedback. I will definitely use this approach going forward.
I agree with Timo that complex structures fail to meet common thresholds for model fit indices in CFA. You can remove some of these items which are loading other dimensions (factors). Don't remove them all at once. Because when you remove one or more, the item and dimension fit can be get corrected. Best wishes, Özgür
A colleague and I have recently conducted research in which we administered a commonly used scale, the Teacher Sense of Efficacy Scale (TSES) to large samples of preservice and inservice teachers in China. Similar to your experience, the items that are commonly purported to load on certain factors did not do so in our research, and, furthermore, my scouring of the literature indicated that the items on the TSES often don't load on the purported factors in other research.
I now suspect that it's possible that some well-known scales are not all that they're cracked up to be in terms of a stable factor structure, and that researchers often simply do not check the factor structure of the scales they use in terms of their specific participants' data - either by means of exploratory or confirmatory factor analysis. Different participant groups yield different patterns of responses - perhaps because of their cultural and experiential backgrounds.
I'm not sure how this might be helpful for you. Certainly, some of the suggestions above are worth pursuing. But I wouldn't become glum in believing that somehow your data are deficient. Believe those data!
Thanks for sharing your experiences Robert. It does seem to be something a lot of other researchers are struggling with.
Karl Van der Schyff. You're welcome, Karl. I wondered whether I should have added that, as part of our most recent research, we assessed agreeableness from the Big Five. We need to do a bit more investigating, by our data, for both preservice and inservice teachers are extremely skewed (very few low scores) and the standard deviations are very small, so the agreeableness items don't seem to differentiate between the people in our samples. To me, that indicates a problem with the agreeableness scale (on top of the problem that "agreeableness" isn't really a word - but that's a different issue).
I hope you enjoy trying to tease out the truth in your data as much as I do - always remembering the old adage that there's a danger of torturing data to the extent that it will finally speak to you. I would much prefer to obtain "quality" data in the first place so that no torturing is necessary.
- Badges
- Science topic
- Similar topics
- Linguistics
- Philology
- Language
- English