I want to compare the effect size for comparisons made at two different points in time. I don't have the original data for one set of time points hence why I can't just compare the actual scores using a multi-variate analysis.
I have effect sizes, M, SDs, and Ns for independent sample t-tests at two time points. I want to show that the effect size has remained the same/has not remained the same over time. How can I do that?
I think that you might be able to use the matafor package in R with the rma.mv function to do this. You could perform a meta analysis on the two effect sizes and with the rma.mv function account for the non-independence of the two effects. I am not entirely familiar with the rma.mv function, but I do think that you can specify that the sampling errors for each effect size are correlated; or indicate that the effect sizes are autoregressive. Either way, your I^2 value should give you some indication of the heterogeneity of the effects...if it is large, you can conclude that the two effect sizes are different. The associated Q test can tell you whether this heterogeneity is significant.
That being said, wouldn't the test for this just be the F for the interaction in a mixed-ANOVA? You have a 2 Group (1 vs 2) x 2 Time (1 vs 2) mixed-design. Do you have enough information to calculate all the sum of squares you need for the F test? I seem to recall that you can calculate SS from means, SDs, and Ns. If the F test is significant for the group X time interaction, then the effect sizes differ, right?
The data is multivariate actually, which is the issue I am running into. I've been looking for ways to calculate SS by hand knowing just the Mean, SD, and N but to no luck thus far. Your R answer I am looking into now. Thanks for the help (again)!
I have a software called Gpower installed on my computer, but I am still not able to show to you how to use it.
- Badges
- Science method
- Similar topics
- Mathematics
- Statistics
- Statistical Modeling
- Analyses of Variance
- ANOVA