Hello everyone,

I'm looking for some advice on my study design. In short, I want to study the impact of mobile VR on a specific skill in an engineering course (are students who study with VR do better as measured by a pre-post test + psychological assessments, such as self-efficacy?). I have 3 classrooms (undergraduate college level), two of which will have about 30 students and one - probably around 15. The semester- long VR intervention is integrated with the existing curriculum (which means we have certain constraints on the order of topics that we do in VR). We have overall 10 modules, all of them can be potentially taught in VR, but we can also teach less than 10. So the treatment is the use of VR (as opposed to using 2D materials for instruction).

Now, the problem is - I would typically use a between-subject design, comparing Classroom 1 to Classroom 2. However, I'm not sure what to do with Classroom 3 AND the problem is that one classroom is likely to be formed by students from a non-engineering program (who tend to be in the same class due to their scheduling), while the other classroom will have engineering students. In other words, there is a problem of baseline equivalence of the two conditions (if one class does VR and the second class does 2D).

If I use within-subject design, then there is a big problem with counterbalancing; some modules have to go first, and alternative VR vs. 2D in other modules between classes causes problems in interpreting the unique contribution of the VR in the outcome variable (which is a test measuring the skill of interest) because the test can only be administered twice (due to the practice effect).

Which design would be more appropriate? Between-subject design is what I would do, but then there is the selection threat to the internal validity; in the within-subject design, teasing apart the effect of VR as opposed to the order of topics and the content of topics is almost impossible. I would appreciate any thoughts about this issue.