I have is a small dataset of 150 observations (villages) and quite a complex model with 6 variables and 13 paths. This results in 23 parameter estimates (q) (including coefficients, constants and variances) from our SEM, which means we have about 6 observations per parameter. This is a lot less than the recommended N:q ratio of 20 observations per parameter recommended by Kline (2015).
Furthermore, ideally we would be controlling for some potential confounders in most of the pathways, so this reduces the N:q ratio even further. When I do this (i.e. run the SEM with extra covariates), the model shows some pathways being highly significant. When I re-run the model without the covariates, these are no longer significant.
I am not sure which of these outputs is more reliable - on the one hand, the N:q ratio in the model with all controls is tiny, so the significant parameters may be completely unreliable. However, inclusion of the covariates may be improving the model by removing/absorbing variance.
Can anyone advise on how I might choose between models?
Thanks in advance!!
It is suggested to estimate 15 study units per variable
Hair, Andersen, Tathan & Black. (1999)
Hello Tanya,
Yes, if you change a model (here, inclusion or exclusion of "control" variables), the results can differ. That's true for any kind of data analysis. I would not lay the difference at the feet of "small sample size," though of course, more cases per estimated parameter is better than fewer cases per estimated parameter.
I'd opt for the inclusion of covariates model, regardless of the results, so long as you can justify the choice of the covariates in the first place. If you just "threw them in" because you happened to have those scores, then that's another matter altogether.
That is, this decision should be just as thoughtful and defensible as the model you chose to evaluate in the first place, since it is a part of the model.
Good luck with your work.
Hi David,
Thanks for your advice - it's how I would see it but being new to SEM, I'm a little concerned about reviewers throwing out my findings because the small sample size might lead to unreliable esimates -
- Badges
- Science topic
- Similar topics
- Mathematics
- Statistics
- Sampling Design
- Sample Size