I'm working on a review of a few experimental studies and I want to produce a forest plot depicting the effect sizes (Cohen's d) and their confidence intervals. However, the only available information is the effect size (d), the sample size, and the p-value.
Most of the effects are calculated from chi-square tests, but not all of them. I want to avoid using odds ratio as a measure of effect, if possible.
Any suggestions?
Effect size is a function of the means and sd for the treatment and control groups (in addition to their N). If you don't have the means/sd I don't see how you can estimate the CIs (or even back into it through reverse engineering).
If you have proportions as the outcome and you have both groups' proportions and N, you can estimate the CIs.
Ariel, thank you for your answer.
In that case, is it at all acceptable to draw forest plots without the CIs of the effect sizes? Would it be sufficient just to indicate whether an effect is significant or not?
I would just create a table with the effect size and N. The forest plot loses it's meaning without the CIs.
You might take a look at the R package "compute.es". It calculates effect sizes and their confidence intervals. For chi-square (1 degree of freedom), all you need is the chi-square value and the number of observations. I don't know how it works, but it may be helpful.
http://cran.r-project.org/web/packages/compute.es/compute.es.pdf
In some simple cases the CI for d depends only on d and sample size so it might be possible depending on design. However the CI is computed differently for different designs and it therefore matters how that was done. In addition the chi-square to d calculations are generally approximations and therefore might also influence the calculations.
Than you all for your suggestions.
Apparently, the best thing to do is to go through the studies once more and calculate the effect sizes and their CIs properly. I have done just that.
In summary, it took me less time to go through the studies one more time than the time I spent looking for a formula to compute CIs just from d and N.
You can do this provided it is reasonable to assume that your estimator of the effect size is normally distributed. Suppose your p-value is .01, then the z-statistic corresponding to the (2-sided test) p-value is 2.58. But, z is nothing but d/se(d), where se(d) is the standard error of d. So, we can compute se(d) = d/2.58. Once we have se(d), it is trivial to do the 95% confidence intervals: d +/- 1.96 se(d). You may note that we didn't even need the sample size! One caution: you should pay attention the to the scale of effect size. If you are calculating Odds-ratios (OR) or relative risks (RR), these are almost always done originally on the log scale, but then exponentiated while reporting. In this case, you calculate the se(d) on the log scale, form the 95% CI on the log scale, and then exponentiate the interval endpoints.
Since it seems you have simply recalculated the effect sizes and CIs, this response is no longer relevant, but I am putting it here for future reference.
The other method of calculating confidence intervals is to use a boostrap/Monte Carlo method, rather than explicit formulas. In that case, all you need are the D values and the number of observations (N). You can randomly permute the N observations, repeat this process an arbitrarily large number of times, and calculate the confidence intervals using the estimated variances of these permuted data sets. In fact, this method is preferred in many contexts (for instance, competing risks survival analysis) where the explicit algebraic form of the CI is either complex or undefined.
You can see Wikipedia for a couple of common methods for doing so, along with references:
https://en.wikipedia.org/wiki/Bootstrapping_(statistics)#Methods_for_bootstrap_confidence_intervals
- Badges
- Science topic
- Similar topics
- Medicine
- Physiology
- Biosignals
- Biomedical Signal Processing
- Statistical Analysis