09 September 2018 6 7K Report

Hi

I have two groups (males = 420 and females = 763) which have taken an instrument we have translated into a new language. We are attempting to validate the instrument. Overall, we found modest fit during CFA for the combined group (likely due, in part, to 2 of the 32 items having below .3 loadings - the rest were great). Those two items remained poorly loaded across multiple CFA iterations (single latent, correlated oblique, and bi-factor), but were significant. My co-author and I are in disagreement about if those items should be dropped and the model re-run. It is my belief that doing so would provide both the basic instrument fit but also the fit as we would recommend it be interpreted in the new translated version as such poorly loaded items offer very little interpretive information. In general, the fit is similar to that observed previously.

When we conduct invariance testing it fails immediately, and badly (results pasted below). My co-author and I are also in a disagreement about how we can interpret this. It is my understanding that the unequal sample sizes will not cause us to be more likely to find between group variance and that, in fact, it makes it more likely for us to fail to reject the null (e.g., that the groups are invariant). Simulation studies I have seen (Yoon & Lai, 2017), and other references to sample size considerations, seem to support this. My co-author remains concerned that our sample size has caused this and that we cannot interpret our results as indicating non-invariance for males and females.

Two questions

====

1. Anyone know a reference to a test development book that says its ok to drop poorly loaded items during instrument translation?

2. Any thoughts, references, or interpretations of the measurement invariance issue as it relates to sample size differences?

Observed Configural Results

====

Chi-Square Test of Model Fit

Value 15611.280*

Degrees of Freedom 460

P-Value 0.0000

Chi-Square Contribution and P-Value From Each Group (degrees of freedom = 230)

MALE 3825.047 0.000

FEMALE 11786.233 0.000

RMSEA (Root Mean Square Error Of Approximation)

Estimate 0.236

90 Percent C.I. 0.233 - 0.239

Probability RMSEA

Similar questions and discussions