I'm in the planing phase of doing a trail for a new therapy treatment for generalized anxiety with 3 groups.
A treatment as usual as usual group, and a wait-list control, both serve as a control for a third group who will be receiving the new treatment.
Here is where i need help;
I cannot figure out what statistical test to use, or even how determine the appropriate sample size when i have 3 groups and not 2.
The idea is to determine how the new treatment compares to treatment as usual, and use the wait-list control group to account for any changes that can be due to environmental changes, such as weather, of historic events.
So you assume (or you think that there might be a risk) that there is a time effect that may imact the outcome in parallel to the treatment effect. If this is corerct, then you need an independent control group to estimate the time-effect. You then will have a 2x2 design with the factors time (before and after) and treatment (control and treated), and your goal is to estimate the differential change over time between the treatment groups. With your design (3 groups, with missing "before/control" group) will underestimate the standard error for the differential effect.
Another name of this particular design is "BACI" for "before-after / control-impact".
Which test is appropriate depends on the assumed distributed of your outcome/DV. You did not write a word about this, so it's impossible to give you a hint here.
To estimate an approriate sample size requires further information about the noise and the effects you (reasonably) expect.
Jochen Wilhelm Thank you. To clarify the wait list control group is because there is a seasonal component to anxiety, its also sensitive to things in society, like when the corona virus broke out, patients with anxiety tended to have reduced symptoms. So if i only had a control and an intervention group i would run the risk of a false positive if both groups showed an effect on anxiety that might actually be attributed to factors in society. So be using the wait list control as a baseline of comparison my thoughts were that i can somehow account for factors outside my control (i cant predict an outbreak). The waitlist should show a difference in symptoms correlating to these outside factors.
Yes, you should (you must) account for changes over time that are not related to the treatment. What you account for are changes. Knowing the mean response off a control at the end of the time period does not tell you the change. Using the mean of the treatment group at the start of the period means to use the same data twice, so you will underestimate the standard error of the difference in changes. And in the treated group, you use paired values (before and after), but the "wait list control" are independent. I don't know if there is a way to get a correct estimate for the standard error of the "difference in change" in this experimental design.
Jochen Wilhelm Thank you again. I'm obviously not very good at statistics. Im not sure I follow, the 2*2 idea. Do i need an extra group?
Here is the basic idea. There is a new therapeutic form gaining some traction in psychology, literature reviews suggest it could be help full for anxiety. I want to test this theory. The setup looks like this.
Treatment is 8-10 weeks. This is comparable to cognitive behavioral treatment patients typically receive. So I want to know how the new treatment compares to treatment as usual on a number of factors: (anxiety levels, memory impairment, attention, and level of mind wandering, along with a scale for the new treatment). I recruit all participants from the same population, which is the actual waitlist for treatment- there’s a waitlist for treatment currently at about 12 weeks. The idea is that since I recruit from the waitlist both groups, and the waitlist will go through the same environmental and societal changes, and the rationale was to simply measure the same factors in both groups and in a comparative sized group in the waitlist. The outcome measure are the scales, most of which (I think, all) are Likert scales. Does this make sense?
So you have patients that have some level of anxeity and after 10 weeks of treatment, this level will change (so you have "before" and "after" data from each patient). You are concerend that this change may not be due (only) to the treatment, but (also) due to other things happening during this time.
To control for everything but the treatment that may contribute to the change, you need to estimate the change in a control group that will be as similar to the treatment group except for the part of the treatment that is of interest (e.g.if the treatment is a series of talks about particular topics or in a particular style, then the repeated talks, irrespective of the topic or the style, have some inpact that is not scientifically interesting. So the control groups should undergo the same number of talks, but on different topics or in a standard style). So for the contrl group you also have "before" and "after" data (also paired) and you can use this to estimate the treatment-independent change. These changes can be compared between the groups. This is equivalent to the interaction (of treatment and time) in a two-factorial repeated-measures design.
Likert scales have the problem that they are clearly not normal distributed. The assumption may be okayish when the values are not at the extremes. I'd make use of the assumption that normalized scales should be approximately beta-distributed (and use a beta- or a quasi-binomial regression model). But I think this is quite uncommon in psychology.
- Badges
- Science topic
- Similar topics
- Mathematics
- Statistics
- Sampling Design
- Sample Size