I applied the same questionnaire in two samples (150 and 60 participants) from different regions from Brazil. I would like to join those samples, but I can not do measurement invariance (because of the small sample size) to evaluate if the construct is equivalent. Can I join the samples with no tests of measurement invariance? Can I test Levene, check if the variance of the constructs are equivalent and then join the samples?
Matheus Perazzo. It might be necessary for you to provide a bit more information before you can get really useful answers. For example, what were you measuring, and what kinds of analysis do you think you can't do because your samples are too small?
That aside, maybe you could do some tests such as t-tests to assess whether your samples are sufficiently similar to be validly combined.
I recommend that in the total sample you use SEM MIMIC, these models are the same as the AFC only they add to the covariates as predictors to determine if there is any effect on the latent variable and even allows the differential analysis of the item (DIF)
Thank you Robert Trevethan for your prompt reply,
I validating a questionnaire about positive health. The samples (mothers) were collected by universities from different regions of Brazil (south and southeast).
I would like to do measurement invariance - MGCFA; however, my sample is not enough for this (150 and 60 participants).
I want to test the external validity of this questionnaire with other diseases, so I intend to do regressions and correlations. Therefore, I would like to combine the samples.
In some validations, the construct are significative different between gender, even though, the questionnaire is validated combining both genders. In my case, if the t-test finds p
Hello Matheus,
Things can be a bit tricky - and it depends on how sophisticated and "careful" you need or want to be. Furthermore, measurement invariance has different meanings, so it's necessary to be clear about the specific kind of measurement invariance you are concerned about.
At a simple level, I guess you could see whether the two groups are different on some important demographic variables. Then, if they do differ, I guess you need to think carefully about whether that matters or not.
Apart from that, I'm sorry, but I don't know what you mean by "external validity" in your context. It is not one of the conventional forms of conducting a validity assessment of a scale, but maybe you're not investigating the validity of a scale.
The term "external validity" has more than one meaning. For example, it can refer to whether findings in one context are generalizable to other contexts. I'm not sure whether you mean that or not when you use the term external validity.
Feel free to get back. At the moment, I don't know enough about your specific investigation to feel confident about making additional comments / suggestions.
Thank you, Robert Trevethan
I appreciate your idea about comparing sociodemographic variables. I am going to test it.
Indeed, I was not clear about external validation. I would like to say that I will test my construct against other variables (diseases, well-being, etc).
Warm regards
All the best with your research, Matheus. If you want to chat about some things along the way, feel free to get back.
- Badges
- Science topic
- Similar topics
- Mathematics
- Statistics
- Sampling Design
- Sample Size