I am trying to semi-replicate a paper and it seems like I can calculate their partial eta squared given the F test values based on this reference paper: https://doi.org/10.3389/fpsyg.2013.00863.
Let's assume that I finish my semi-replication. Then, I can compute an adjusted R squared (because I am using lmer not anova) for my model. Can I then compare my adjusted R-squared with their partial eta squared to see which one has a larger effect size? Or is this comparing apples to oranges?
Hello Pardis,
Apples and oranges, yes.
If the original paper was reporting partial eta-squared based on a single factor/independent variable, then that value is numerically the same as an ordinary eta-squared, and would be equal to the unadjusted R-squared from treating the same dataset as a linear regression model. However, it would not be the same as the adjusted R-squared (which will be smaller by some amount that depends on number of predictors relative to sample size).
Good luck with your work!
You have another problem here. When you publish the results you will have to write a justification showing that the two are equivalent. If you happen to get reviewers that are fluent in statistical issues this might be easy. For the remaining 90% of potential reviewers this will be a huge uphill battle.
Would you consider running lmer for your data, but also ANOVA so that you have a result from your data that is comparable to previous publication? You could ask for the raw data from the earlier study as an alternative approach.
If you are using lmer then I assume you have multiple random variables or variance components. What do you mean by R^2 (or adjusted R^2) in this circumstance? This is an older publication, but a good place to start:
- Goldstein H, Browne W, and Rasbash J. 2002. Partitioning Variation in Multilevel Models (PDF, 139kB). Understanding Statistics, 1(4), 223-231. It is available at http://www.bristol.ac.uk/media-library/sites/cmm/migrated/documents/partitioning-variation-in-multilevel-models.pdf
ps1. There are a few "adjustment" methods, so you'd want to specify these.
ps2. David Morse , I think I am misunderstanding your answer. The F value is a statistic that can be reported for any linear model, with different sets of different types of variables combined in "linear" ways (i.e., interactions, polynomials, b-splines). Are you saying that you would need something beyond the F statistic and its degrees of freedom to calculate these adjusted statistics.
ps3. @Pardis, I see you work at Stanford. I'll be there in two hours!
Thank you everyone for responding back.
Timothy A Ebert , yes I contact the authors to get them to share their data with me.
Daniel Wright Thank you for the reference. You are right. R^2 or adjusted R^2 consider how much the model explain the variability in the DVs. I think what I meant was the just the estimated effect size of each fixed IV.
Daniel Wright , what is brining you to Stanford? are you giving a talk today?
- Badges
- Science method
- Similar topics
- Mathematics
- Statistics
- Statistical Modeling
- Analyses of Variance
- ANOVA