Hi,
I want to investigate if there are significant differences in the difference of two groups.
I have two populations and both populations have a control group and a experiment group. I want to investigate if there are signifcant differences between the two populations in the difference between the control group and experiment group.
For one of the populations I have the complete data set, but for the other I only have the means, sample size and standard deviations.
How do I go about this?
Thank you in advance.
Best regards,
Aloys
Hello Aloys,
Assuming that differences implies comparing of mean scores, then here is one very simple approach:
Compute Cohen (or bias-corrected, like Hedges) effect size values for each population comparison, with a confidence interval estimate for each (see below for references). This you can do with just summary statistics (as you have for the second population).
If the two C.I. estimates of ES do not overlap, then conclude that the exp-control differences are not consistent across populations. Note that this is a conservative test (e.g., non-overlap implies pretty pronounced distinction).
Good luck with your work.
Simple start for ES CI: https://www.uvm.edu/~dhowell/methods/Supplements/Confidence%20Intervals%20on%20Effect%20Size.pdf
Hi David, Ajit and Jos,
Thank you all for your reply. Do these tests also suffice when the data of one of my populations is not normally distributed?
Hello,
I conducted both a t test and Cohen’s d test and got different outcomes: the t test showed significant differences between the populations, while the Cohen's d test did not. Both of the groups in both populations are normally distributed. Sample sizes for all group are above 70 and the standard deviations do not vary significantly between groups.
Any idea how this difference could come about?
Thank you.
Best regards,
Aloys
I suppose that --for each population-- you took the mean (plus n and sd) of the differences between control and experimental group. If so, you ended up with 2 means (plus n's, and sd's). The online t-test based upon these 2 different means, (plus n's, and sd's) seems correct to me.
If you don't know the standard deviation of the differences, you can compute these using the formula:
The standard deviation of the difference between sample means (σd) is approximately equal to:
σd = sqrt( σ12 / n1 + σ22 / n2 )
see: https://stattrek.com/sampling/difference-in-means.aspx
- Badges
- Science topic
- Similar topics
- Mathematics
- Statistics
- Sampling Design
- Sample Size