You are doing great research and an outstanding science experiment.
As a medical statistician, my advice to you
Firstly: Consider the repeated measures ANOVA test that requires more than two measurement times
At least one quantitative variable is needed to compare the mean between groups.
If you follow up with patients only before / after,
You do not need to use repeated-measures ANOVA
You can use T-test to check the effectiveness of the treatment by comparing the patient's condition before and after
Or its non-laboratory alternative, Wilcoxon, in case the data is not distributed normally
As for the comparison between the four multiple groups, you can during the same time (In the case of after, for example) the normal ANOVA procedure to compare the condition of patients in different groups, including the control group, if any, after taking the period for treatment
As well as for the case of the groups before the start of the experiment. Through the multiple binary comparisons, you can determine the source of the difference in favor of any of the drug groups.
This requires that the data be normally distributed or the sample size is greater than 30 individuals, If not, then use Kruskal-Wallis's alternative ANOVA test.
Second, don't go too far with more complex models because statistically things are gradual
Start with the basic tests, which will lead you if you do not reach your goal towards the higher-level models
If you’re considering using ANCOVA, it’s important to note that a covariate is a continuous independent variable that is added to an ANOVA model to produce an ANCOVA model. The inclusion of a categorical variable as a covariate in ANCOVA is possible and can be useful in certain situations. For example, if you have a categorical variable that you believe may be related to the dependent variable, you can include it as a covariate in your ANCOVA model.
However, it’s important to keep in mind that the inclusion of a categorical variable as a covariate in ANCOVA can lead to some issues. One issue is that the interpretation of the main effects and interactions in the ANCOVA model can become more complex when a categorical variable is included as a covariate 1. Another issue is that the inclusion of a categorical variable as a covariate in ANCOVA can lead to problems with multicollinearity, which occurs when two or more independent variables are highly correlated with each other.
After a first quick look at your question, I wonder if you have considered treating the baseline measure of emotional regulation as a covariate in a 2x2 model like this:
DV = emotional regulation score at follow-up
Covariate = emotional regulation score at baseline
Factor 1 = medication status
Factor 2 = Intervention (CBT vs TAU)
Include the med status x intervention interaction
One possible fly in the ointment is that medication status was observed, not manipulated. Some folks are generally opposed to ANCOVA if the groups are not created by random allocation. Your situation is a bit different form the one that is usually objected to because you do have random allocation for your intervention variable, AFAICT.
Daniel Wright, I believe you generally oppose ANCOVA when the group variable is observational. What do you make of Mya East's study, where one variable is created by random allocation, but the other is observational? Would you consider including the baseline score as a covariate here?
Hi Bruce Weaver . I'm good with ANCOVA in observation, but with the caveat that this fits better (in estimating a causal effect) than gain score model when the covariate influences the group allocation (even indirectly). This is stated most clearly in Article Allocation to groups: Examples of Lord's paradox
. That said, I think two main things are most important. 1. Do a scatterplot and look at is carefully, and 2. think about how the covariates related (particularly in causal terms) with group allocation. But as you say, the ANCOVA is more powerful with the RCT. Also, the gain score makes assumptions about the pre and post measuring the same thing. So, in short, I agree with you on ANCOVA here after looking at scatterplots!
Bruce Weaver Daniel Wright Thank you all, I appreciate the insight! I had read about including baseline/T1 as a covariate and I like the idea of incorporating a 2 x 2 ANCOVA. However, Imad-Addin Almasri mentioning the use of a t-test to compare group differences between treatment and control, has me thinking about the possibility of including T3 that would be a follow-up time point.
Assuming treatment would produce significant results, I believe being able to show results are maintained at follow-up would further strengthen my study and indicate clinical significance. I do not want to overcomplicate my study by introducing multiple factors/times/measurements but I wonder in the case of having 3 time measurements if the original idea of using the repeated measures ANOVA would fit better. However, there still is the idea of measuring medication status which I do believe, based on previous literature, will be influential on results. I wonder if the possibility of including medication status as a covariant in this example is better suited. Or a different method entirely.
I am open to any suggestions on this final thought, even if the original concept of only two time points seems most suitable in this scenario. Thank you :)
A repeated-measures mixed ANOVA is a valuable statistical analysis when you want to assess the impact of an influencing factor within your independent variable while considering repeated measurements. Here's a summarized guide on how to use this analysis:
Define Your Variables:Dependent Variable: This is the outcome you aim to measure. Independent Variable: It represents the factor you believe influences the dependent variable and can have multiple levels or conditions.
Repeated Measures Factor:Identify the factor that is measured repeatedly within each level or condition of the independent variable. This could be time points, sessions, or other repeated measurements.
Data Collection:Gather data for each combination of the independent variable and the repeated measures factor. Ensure that participants are measured at multiple time points or under various conditions.
Perform Repeated-Measures Mixed ANOVA:This analysis assesses the impact of the independent variable (between-subjects factor) and the repeated measures factor (within-subjects factor) on the dependent variable.
Interpret Results: Focus on main effects and interaction effects:Main effect of the independent variable: Indicates if there are significant differences among the levels or conditions. Main effect of the repeated measures factor: Shows whether there are changes over time or across repeated measures. Interaction effect: Helps you understand whether the influence of the independent variable varies across time or repeated measures.
Post-Hoc Tests (if necessary):If you identify significant effects, conduct post-hoc tests to pinpoint specific differences between levels or time points.
A repeated-measures mixed ANOVA is ideal for situations where you have data with repeated measurements on the same participants under different conditions. It accounts for individual differences while examining how the independent variable and repeated measures factor interact to affect the dependent variable.
However, it's crucial to ensure that certain assumptions, such as sphericity and normality, are met before conducting this analysis. If these assumptions are violated, consider alternative analyses or data transformations. Collaborating with a statistician or data analyst is advisable for planning and executing this type of analysis effectively.