I am conducting a study testing the effectiveness of a kind of group psychotherapy. There are 10 participants in my experimental group and 14 participants in my control group. At first, I planned random assignment to the groups, but because of the time of the group therapy, 14 of the participants wanted to be in the waitlist control group. After I created the control group, I run a t test to compare two groups in terms of some study variables. When I did a t test, I saw that there was no significant difference between the groups in terms of the study variables. In summary, the groups have similar characteristics (e.g. Age, educational level, romantic relationship status, mean scores of the participants). However, group sizes are different. Can I do my analysis with 10 people in the experimental group and 14 people in the control group? If no, how do I remove the 4 people in the control group?
The unequal variance t-test is an independent t-test and is used when the number of samples in each group is different, and the variance of the two data sets is also different. This test is also called Welch's t-test :)
Ceren Bektaş-Aydın, please report n, mean, and SD for each of your groups. Thanks.
Two issues, not related to t-test question but worth addressing before getting to that, are:
1. You say "That's why I made a separate announcement for the control group." Can you explain this more? If random assignment could be done but wasn't, and different recruitment was used for the two groups, this is an issue.
2. You appear to treat non-significant results from a series of 10 v 14 comparisons as meaning the non-random assignment isn't an issue and are accepting the null hypothesis on the basis of low powered comparisons. Can you expand on this?
I agree that you could conduct your analysis with the existing group sizes using statistical procedures that can handle unequal group sizes, such as the independent samples t test. However, you should keep in mind that doing so might affect the statistical power of your study. Here is an easy read.
Schwartz, B. M., Wilson, J. H., & Goff, D. M. (2019). An easyguide to research design and SPSS (2nd ed.). SAGE. https://us.sagepub.com/en-us/nam/an-easyguide-to-research-design-spss/book255242
Good luck,
Two groups could've different sample sizes. However, as safe side Mann Whitney non-parametric should be used for such a sample.
Daniel Wright Actually, I expressed the process incorrectly in the question. The timing of therapy was determined together with the participants. Since the days and hours of therapy did not suit some participants, it was announced that they could be included in the waiting list control group. Since the participants were university students, their course schedules were different. For this reason, a common day and time that would suit all 24 people could not be found. When 10 people joined at shared time, the remaining 14 people were voluntarily placed on the waiting list.
There are many experimental studies where the control group is larger than the experimental group. Moreover, they use parametric tests. I would like to know the basis for it and ask if you know any statistical sources about it.
Ceren Bektaş-Aydın , I am reminded of this BMJ Statistics Note:
https://pubmed.ncbi.nlm.nih.gov/19349342/
https://www.bmj.com/content/338/bmj.a3166.long
I am also reminded of what the late David Howell said in his textbook Statistical Methods for Psychology (multiple editions). This is not a direct quote, but he said (with respect to t-tests and ANOVA models) that unequal sample sizes and heterogeneity of variance do not mix. Some authors go so far as to recommend always using the Welch-Satterthwaite (unequal variances) t-test--e.g., see this blog post by Daniel Lakens:
http://daniellakens.blogspot.com/2015/01/always-use-welchs-t-test-instead-of.html
I myself do not go quite that far because:
1) The unpaired t-test is very robust to heterogeneity of variance when n1 = n2. (Some authors have suggested that with equal sample sizes, it will be fine for sample variance ratios as high as 4 or 5.)
2) It is the population variances, not the sample variances that are assumed to be equal.
3) Even if the population variances were exactly equal, the two sample variances would never be exactly equal.
4) I think we should always try to use the most powerful test that is available to us.
Putting those points together, I don't think the Welch-Satterthwaite test is the most powerful test available when n1 = n2 (or nearly so) and the ratio of the sample variances is no greater than 4 or 5. In that case, I would very likely use the ordinary pooled variance version of the t-test.
I hope this helps.
Bruce Weaver Thank you for your answer. The data met the normality and homogeneity of variances assumptions. I read your shared notes and also took several expert opinions. Using paired sample t test does not seem problematic.
- Badges
- Science topic
- Similar topics
- Mathematical Sciences
- Statistics
- Statistical Software
- SPSS