I am working on a quasi-experimental research study. The study aims to see if there is a difference between student scores on clinical skills between a skills course that is taught in two different modalities – intensive format vs fully online format. The issue is the groups are not randomized (selection bias) but also in the fully online format, three different instructors were teaching 5 different sections. Only one teacher taught the intensive format. Ideally, we want to take the 5sessions and turn them into one large group to compare to the other group (i.e., online format vs intensive format) to see if one is better than the other on clinical skills outcomes. There are pre and post-measures of student clinical skills using a Likert scale instrument.
The sample size is about 60 students for the 5 different sections of the fully online format and about 50 students for the intensive format.
I am trying to determine how to have 1 group of the 6 sections for a paired t-test or a non-parametric test even though they have different teachers. What are my options for analysis?
Thanks,
Hello David,
Unfortunately, your ability to test for the influence of course delivery method is severely limited due to both self-selection by students (into section/method) and the complete confounding of a single instructor with the intensive format (e.g., s/he might have been the best or worst possible instructor).
First, decide on the variables: How many clinical skills are being evaluated via self-report? Do you intend to assess these individually or multivariately? How many items contribute to the score for each skill? (Much more than one would be preferred; you'll frequently see multi-item/summated scores being treated as if they are continuous variables in the literature.) Have you and your team checked to see that the skill scores are coherent (e.g., via factor analysis, or polytomous item response model, for example; as well, these methods offer better justification for treating the composite scores as interval strength)?
If you are confident that the scores are defensible as interval strength (see above), it seems to me that using pre scores as covariate/s, and post scores as the DV (with section as the IV) would be the suitable analysis (one-way ancova or mancova, depending on whether univariate/multivariate method made more sense). For the "main effect" of delivery method, a contrast combining sections 1-5 vs. the lone intensive section would be the appropriate analysis.
If all you wish to do is judge whether scores changed from pre to post, and you were convinced that delivery method made no difference, then and only then would it make sense to pool all six sections together for a paired test (whether Wilcoxon or dependent t depends on what you are willing to presume about the scores). Though less powerful, it might be better to evaluate and report such comparisons separately by section.
Good luck with your work.
I see you're an assistant professor. This i would do:
1 find the person who gave you this assignment
2..Dump the results on his desk say
3. You show me how to do this.Sir
4. Your plan was followed exactly.
5. One of two.things will happen
a the person will realize the error and somehow recycle or
you will apply for your new job happily
BTW Something similar happened to me
When I left the end of the year my Dean said Congratulations, shook my hand, and that was the best moment I had there. Some day I will tell you about the provost.
Seriously, if these folks are that bad you're risking your entire career. I never ever looked back I suggest you don't either. Best of luck, David Booth
Your sample size is also 4.
Don't you think that the variations in what instructors do matters?!?!?
- Badges
- Science topic
- Similar topics
- Biological Science
- Histology
- Sectioning