I am conducting a meta-analysis on correlations among some constructs. So far based on what I have read from meta-analysis resource books and the literature, usually we first transform Pearson r to Fisher's z for meta-analysis, and then transform Fisher's z back to Pearson r for final report. Then I got the following basic questions.
1) Is only Pearson r be included in meta-analysis? What about other types of correlations, such as Spearmen rho, point biserial correlation, etc?
2) Refer to the Comprehensive Meta-analysis, it said t-value can be used to convert to correlation r. Which sort of t-value does it refer to? Is it the t-values in multiple regressions results tables, which can be converted to partial correlation coefficient? What about t-value from independent samples t-test, say, on mental wellbeing between male and female?
3) Any well-defined set of quality assessment tool and framework for correlational studies?? like the Cochrane Risk of Bias Tool for randomized controlled trials?
4) I got 4 coders to code the included studies. Generally speaking, how to calculate the inter-coder reliability of their codings?
Any advices and resources would be appreciated. Thanks all.
1) No - methods exist for other correlations. For point biserial one would normally use a d family metric as this often reduces to a difference in two means
2) There is no one answer. One can convert a t test to an r value but the correct conversion depends on design, balance in groups and so on. Similar concerns exist of regression where one needs to think about presence or absence of covariates etc. the goal is to have a metric that captures the same effect and if it doesn't to handle that elsewhere in the model (e.g., moderators) or exclude those studies
3) There are standard guidelines such as PRISMA that are worth looking at.
4) Not sure. Various methods exist - see link to Uebersax's web pages.
http://www.john-uebersax.com/stat/agree.htm
1. A point biserial correlation is just a Pearson's r computed on a pair of variables where one is continuous and the other is dichotomized. Similarly a Spearman's rho is simply the Pearson applied to ranked data. So, they are, in fact, Pearson's coefficients and I don't see why you might be precluded from meta-analyzing such coefficients.
The point-biserial and the phi will not attain full range unless the dichotomization is 50-50 for the dichotomous variable(s) and it is possible to introduce a moderator into your analysis if you mix coefficients. However, this is unlikely, particularly if the coefficients are generally modest and the splits are not too extreme (gender would rarely be very extreme, for example). If you like, you could manually correct the point-biserial (by computing a biserial). You could correct the phi coefficient by computing tetrachorics (Ubersax has a couple programs to compute tetrachorics on the web page Baguley posted above). I don't think you can nor need to correct the Spearman's r.
2. The t that you get from a paired-sampled t-test is comparable to the Pearson's r that you would have gotten by correlating the outcome measures for the two paired samples. So, these can be freely converted and combined.
You can also convert a t arising from an independent sample t-test. However, I think it's a mistake to comingle r's arising from independent-samples t-tests and r's arising from paired samples (whether the r was computed directly or converted from a t). The issue is that these designs have different power. If you do use comingle these, I would input the sample size of the r's computed from independent samples t-tests as the smaller of the two group sizes. A far better approach would be to analyze these studies independently as standardized mean differences (or to omit these studies from the meta-analysis).
You should not convert t's arising from other situations, like t's reported in tests of the regression coefficients. It makes no sense to convert such t values into r and then meta-analyze.
I believe that the standardized regression coefficients are (or are essentially) partial correlation coefficients. I wouldn't meta-analyze them unless regression models from which you are sampling weights are the same across samples (same dependent variable, and same predictors). Even so, I imagine the variance estimates will be wrong (I think they would be different for independent correlation coefficients vs. dependent partial correlations) and the population estimates will be estimates of partial correlations, not correlations (i.e., under estimates).
Just to be clear: I don't think regression output is particularly amenable to meta-analysis. If the research followed best practices for reporting results, then they would have supplied the zero-order correlations and you can use those in meta-analysis.
3. I don't understand the question.
4. If you had them code the same studies, then you have many choices about how you analyze the overlapping data. If the codes are all ordinal/numeric, then you can compute correlations (there will be 6 intercorrelations among 4 coders) or an interclass correlation coefficient (which will be approximately the same as averaging the six intercorrelations). You could also compute the percent of absolute agreement across all four coders for each variable.
If you did not have them code overlapping sets, then you have few options. In that case, I would have them recode some overlapping studies, or do so yourself, or recruit a new coder to do it.
Thanks both of you, Thom and Alan, for the detailed explanations! Just some follow-up questions below.
1a. Does it imply the Pearson’s r, point biserial correlation, and Spearman’s rho coefficients can be directly coded and meta-analyzed?
1b. I don’t fully understand the following part. Would you mind further elaborating it?
//The point-biserial and the phi will not attain full range unless the dichotomization is 50-50 for the dichotomous variable(s) and it is possible to introduce a moderator into your analysis if you mix coefficients. However, this is unlikely, particularly if the coefficients are generally modest and the splits are not too extreme (gender would rarely be very extreme, for example). If you like, you could manually correct the point-biserial (by computing a biserial). You could correct the phi coefficient by computing tetrachorics (Ubersax has a couple programs to compute tetrachorics on the web page Baguley posted above). I don't think you can nor need to correct the Spearman's r.//
2. I don’t fully understand the following part. Would you mind further elaborating it?
//The issue is that these designs have different power. If you do use comingle these, I would input the sample size of the r's computed from independent samples t-tests as the smaller of the two group sizes. A far better approach would be to analyze these studies independently as standardized mean differences (or to omit these studies from the meta-analysis).//
1a&b. Spearman's rho can directly be coded and meta-analyzed.
Point-biserial correlations are Pearson coefficients computed on variables X and Y where one variable is continuous and the other variable is dichotomous. You could choose to directly code and meta-analyze point-biserial correlations. The issue with the point-biserial correlation is that they are systematically closer to zero than the Pearson coefficients computed on continuous X and Y. That is, the dichotomization of one of the variables attenuates the correlation (i.e., makes the magnitude of the correlation smaller). The degree of attenuation is probably too small to be an issue, but it's hard for me to be sure in your particular case.
So, the most technically correct thing to do would be to correct the point-biserial by estimating the Pearson coefficient computed on continuous X and Y. The corrected coefficient is called a biserial; it's an estimate of what the point-biserial would be if the dichotomized variable were not dichotomized. The first attached link discussed the point-biserial and the biserial.
You didn't ask, but in addition to the point-biserial, there is the phi coefficient, which is a Pearson coefficient computed on X and Y which have both been dichotomized. This produces greater attenuation. The estimate of the Pearson coefficient with continuous X and Y (from a pi coefficient) is called the tetrachoric correlation.
For both the point-biserial and the phi coefficient, the degree of attenuation is greater when the dichotomized variable has a split that differs from 50-50. So, if the point-biserial correlation is computed on dichotomous X and continuous Y and 50% of the cases have X=1 and 50% have X=2 (or other coding, the actual values don't matter) the attenuation is smallest. As the split diverges from 50-50, the attenuation becomes worse. If the split were 95% - 5% then the biserial = 2.9*(the value of the point-biserial). The tetrachoric is harder to compute than the biserial and I'd recommend software designed for it.
2. Don't meta-analyze studies using different designs (unless you know that they are comparable). See the Morris & DeShon study in the second link.
A simple solution would be to perform independent meta-analyses on studies using independent and paired samples. The paired samples can be converted into correlations and combined with other correlations from similar designs and meta-analyzed as correlations. The data from independent-samples t-tests can be converted into standardized mean differences and meta-analyzed together. But don't combine them in one analysis.
http://ncss.wpengine.netdna-cdn.com/wp-content/themes/ncss/pdf/Procedures/NCSS/Point-Biserial_and_Biserial_Correlations.pdf
Article Combining Effect Size Estimates in Meta-Analysis With Repeat...
Gloria Yuet Kwan Ma
I'm encountering the same puzzle.
Some literature supplied a procedure different from Dr. Alan : first tranform Spearman rho to Pearson r, then transform Pearson r to Fisher's z for meta-analysis, finally transform Fisher's z back to Pearson r.
How did you finally pooled the raw data and evaluted the quality of the included studies?
Thanks a lot
- Badges
- Science topic
- Similar topics
- Geoscience
- Cartography
- Mapping