I set up a 10 week intervention for medical students to improve their nutrition knowledge (14 MPC-questions), attitudes (12 questions, 5-point Likert), and self-efficacy (10 questions, 5-point Likert). Students could apply to participate in the course, so the intervention group was not randomly selected. We developed a questionnaire that we distributed pre-intervention and the same questionnaire was distributed post-intervention (n=130). A control group was given the same questionnaire at the same two points in time, however, characteristics between the intervention and control group differed (for example, the ratio MSc/BSc students differs between groups). What will be the best statistical method to analyse the results? I am currently thinking about a paired t-test (still need to check normality of results), but then it will not be possible to correct for potential confounders. Looking forward to your recommendations + explanations!
Hello Hester,
It might be wise to use the pre-intervention measures as covariates, and compare the two groups on "adjusted" post-intervention measure scores. You may consider whether it makes sense to treat the three scores as separate DVs (with three analyses), or as a single, multivariate set (and do one mancova instead of three ancovas).
You are correct in suggesting that the design you used isn't the strongest; randomizing cases to treatment condition would help make a better argument that any observed differences were a function of the intervention and not extraneous influences. As well, random selection of study participants from a well-defined population makes a stronger case for generalizing findings beyond your study sample.
Good luck with your work!
Hi Hester,
You may want to consider propensity score analysis, either weighting or matching, to create a population that can be measured with a t-test as if they had been the result of a randomized experiment.
One of the assumptions of propensity score analysis is that you have access to all covariates that have an effect on treatment and outcome and I do not know whether that applies in your instance.
If you are relatively certain you have access to all necessary covariates, propensity score analysis is probably the most well-studied and regarded methodological option to attain results as close as possible to what would have been obtained if you had a conducted a randomized experiment.
Best,
Aran
- Badges
- Science topic
- Similar topics
- Mathematics
- Statistics
- Applied Statistics
- Survey Design
- Questionnaire