Sorry, I meant r from '-1 to 1', not from '0 to 1'. But the same question remains.

Mikhail,

It might be helpful to articulate exactly what you are trying to determine by examining effect size. Are you concerned that your calculated ES may be too high or too low to be correct?

The correlation coefficient r for any two variables, when calculated correctly (through a standard program like SAS, SPSS, or even Excel), can NEVER by definition fall outside the -1 to 1 range. There is no need to round up or down from outside that range.

I would suggest checking your formulas or whether a correlation coefficient is really what you need to calculate with regard to variance values - you must have an entire set of data for two or more variables to calculate r.

Variance for any data set, of course, can be very high or very low. This also applies to a lesser extent to SD values, depending on the scale of your values, although for a normal distribution SD is rarely outside 3 or -3.

Thanks Neil

I realize, it was imprecise. Let us forget about effect size... Suppose we have one sample and data available are: means at baseline and follow-up and mean of change along with their standard deviations. All continuous. We'd like to know what is the correlation between follow-up and baseline variance. Cochrane Handbook suggest that we use the formula I mentioned. Chapter 16.1.3.2 (http://handbook.cochrane.org/chapter_16/16_1_3_2_imputing_standard_deviations_for_changes_from_baseline.htm)

If, e.g, SD of change is much wider than SDs at baseline and follow-up, than we end up with r much below -1. 

I don't know, maybe Fisher z transformation should be applied? I'm not a statistician, so I'm quite lost here.

This r is important. E.g. beta coefficient (slope) may be calculated based on this r and used for ANCOVA when obtaining the effect size.

Thank you for including the article. That helped a great deal.

While Pearson's Correlation Coefficient ranges from -1 to 1, I don't see what constrains this value similarly. I went to Excel, made a couple of columns of random numbers and  got a CorrE of 2.499 comparing 1000 values from a uniform distribution against 1000 values from a normal distribution.

That said, the closer I read Chapter 16.1.3.2, the less comfortable I was with what it was suggesting. It is also possible that the formulae were not printed correctly. Can you find this method elsewhere? I humbly suggest that there are major problems with this method.

The question (as asked) is complicated. Typically we are interested in means of some value. Here we are interested in variances. So how have you replicated your observations for variance? If you sampled 1000 patients, that is nice. Yet that will only give you one sample for variance. Your choices are to take more samples of 1000, or to subsample your data in some way (key words: computer intensive methods; resampling methods; Bootstrap; Jackknife). You could get multiple estimates of variance by combining different studies.

The correlation between two variables is the cosine of the angle between the two regression lines: y on x, and x on y. If you interpret the SDs in the formula you gave, it is the cosine rule. Therefore the formula seems to be correct.

However, it is only valid if all the standard deviations come from the same paired dataset. As I read the paper, it seems to be about using published studies when only summaries of the data are available.

It seems likely that SDchange does not come from the same source as the other SDs. Perhaps you could give a little more detail about the data.

PS: I don't know what you mean by 'correlation between the variances'. Correlation is a measure of strength of association between the pairs of observations assuming the relationship is linear. If the relationship is non-linear, correlation can be high or low (e.g. if y = x2 and X is uniform between 0 and 5, the correlation is high, but if X is uniform between -5 and 5 it is zero).

My special interest is in meta-analysis. Terry , you are right, I always have only summaries available. RCTs usually report means and SDs at baseline and follow-up and sometimes means and SDs of change. There are 3 basic approach to compare the effect of intervention: 

1) Compare final values

2) Compare change score

3) Compare final values adjusted on baseline scores

The third approach is the best. But it is ANCOVA. As ANCOVA is a regression model, I have to calculate slope (beta1). That calculation requires a correlation coefficient final/baseline. The only formula I found from the Cochrane Handbook is aforementioned one (this formula is shown in almost every book on meta-analysis). 

When I am thinking about this formula, I feel lost. In Pearson squaring is done in denominator. Instead, here it is done in numerator. Additionally, there is a third variable SDchange in numerator but not in denominator. So, mathematically it is clear that if SDchange is considerably higher than SDpre and SDpost than r < -1. And if e.g. SDpre is considerably higher than  SDpost and SDchange than r > +1.

Try e.g. numbers: 

1) SDpre = 2; SDpost = 4; SDchange = 10 (r < -1)

or 2) SDpre = 6; SDpost = 1; SDchange = 1 (r > 1)

Mikhail, you are correct, the numerators are quite different, but both formulae have the product of the standard deviations in the denominator.

In the usual correlation formula, if Dy and Dx are deviations of dat points from the means then 2sum(DyDx) = sum(Dy2) + sum(Dx2) - sum((Dy-Dx)2) [binomial formula] from which we derive the formula you gave.

So I think the method is to plug in crude approximations to these values. The paradox must be due to the approximations being not good enough.

So I wonder if that is a fault with the method, or you are applying it incorrectly.

Notice the similarity to the cosine rule.

If one could identify the two regression lines (which cross at (meanx, meany)) and take another point on each line, then the cosine rule gives

• (a2 + b2 - c2)/2ab

where a, b and c are the sides of the triangle formed by those three points. If we could identify those sides with the standard deviations we would get an approximation to the correlation.

Thank you Terry! I think, I got it now. Certainly, you are right. SDpre is an adjacent, SDpost is a hypotenuse, and SDchange is an opposite. I see now, that my question was wrong (too theoretical) as there is no way to break Pythagorean theorem -> SDchange^2 cannot be larger than SDpost^2. So, in theory, we can play with any numbers in this formula, but, in real world, geometry sets its own limits. There is nothing wrong with this formula. I'm very happy. Thanks again. :-)

That's good. I didn't realise that the data were not from actual studies. There could still be a problem if different components come from different studies when these are slightly contradictory. I don't know enough about meta analysis to understand how we choose the data to plug into the formulae. Are they averages over different studies? You have to have some way to ensure consistency.

You could still find that a confidence interval for r might not satisfy the constraints. When that happens the model has broken down. This can occur with a confidence interval for a proportion if p is close to 0 or 1. In that case the normal approximation to the binomial is not accurate enough. In your case, it won't happen if you can find the exact distribution of r^, but it can happen with some approximations.

Actually, Terry, I was thinking it would be more precise to say, that SDchange should be between SDpost - SDpre ja SDpost + SDpre. As the triangle is not always the right triangle.

The data example is fictional. Unfortunately, I also ran into this problem many times for real. 

They are not averages over different studies. I need these formulas for "preparing" the first phase of meta-analysis. I calculate adjusted means and variance for each study separately, and then put them into a software for pooling (usually CMA). When in CMA, I can run some sensitivity tests to ensure that at least approximately the synthetic result is right.

Yes, I realised that you were talking about the special case of a right angle (r = 0).

When the problem happens with real data it shows inconsistency. I suppose that's a hazard with such data sources. In that case you have to decide whether to ignore the study, round to -1 or 1, or find some other fudge.

I have exactly the same problem as you. I am wondering how did you solve this? Thank you!

If you constrain your y-intercept to be zero (0), then the slope of the relationship (often interpreted as the correlation) can (and often will) be great than +1 (or less than -1). However, in doing so, you're not minimizing the sum of squares of the residuals and therefore not calculating the correlation coefficient. You must calculate the effect size of the regression including a constant term (i.e., R^2). The correlation coefficient is then the root of this squared term.