I am planning a study that explores the impact of a sudden change in the way learners are taught on their executive functions and academic achievement. I plan to measure the outcomes (executive functions and academic achievement) three times before the change as well as three times after the change. In addition, there is a matched group who do not go through this change. I am unsure which statistical test would be best to analyse the resultant data.
A linear mixed methods regression model seems appropriate, but I think it is out of my ability range. Also, I do not understand if I need to average the test scores before and after the change for the two groups of learners?
The repeated measures ANOVA is also an option, but I am uncertain.
Advice would be most welcome.
Jaqueline Harvey A linear mixed-effects model will allow you greater flexibility as you will be able to examine fixed factors, random factors, and crossed factors within a single model (unlike rANOVA).
To your second question - No, you do not need to average the raw scores first to run the LMM. The model will obtain the slopes and intercepts from the raw data.
I recommend researching how to conduct a LMM in whichever software you plan to use for the analysis. If using R, the lme4 package works well.
While I understand your reasons for wanting to do an ANOVA, I would suggest that you consider the LMM as an alternative.
Given your repeated measures design (n = 6 observations per subject), it is very likely that you will encounter missing observations (e.g., subjects not getting back to the survey, missing lab appointments, measuremt errors during the experiment etc.). While you can of course outweigh the loss of data from ANOVA (because ANOVA assumes that there are the same number of observations per subject) by aggregating means, you will lose valuable information about variation and patterns in the data.
In addition, the compound symmetry assumption of ANOVA is often too restrictive. Repeated measures tend to be positively correlated - that is, observations that are closer together tend to be more similar than those that are further apart. Furthermore, it is not uncommon for variance to increase over time (some subjects may be more responsive to the treatment than others - i.e., violated variance homogeneity). For the latter, you might consider Mauchly's test, which is included by default in the ``anova_test'' function in the ``rstatix'' package. However, Mauchly's test has some limitations, as it is sensitive to sample size.
Having said that, I recommend the use of LMMs; they are not as intimidating as they may seem at first glance. You can find a lot of information online, especially for the ``lme4`` package. Alternatively, you might consider using JASP, an open source software that is user friendly and intuitive.
Good luck!
- Badges
- Science topic