The OLS regression, y ~ x, assumes the x values are measured without error, so y and x are not treated the same. You my look into total least squares regression (which has other names too, like orthogonal, Deming, principle component regression).

That's true. That is why the confidence intervals for regression coefficients and correlation coefficients are different. However the algebra shows a relation between the 2 ESTIMATES but not the confidence intervals UNLESS you consider bivariate regression where the assumptions are different. See: https://www.google.com/search?client=firefox-b-1-d&ei=196PX7fOG8nOtQblpYb4Dg&q=bivariate+regression&oq=bivariate+regression&gs_lcp=CgZwc3ktYWIQAzIHCAAQyQMQQzICCAAyAggAMgIIADICCAAyAggAMgIIADICCAAyAggAMgIIADoECAAQRzoJCAAQyQMQFhAeOgYIABAWEB46DggAEOoCELQCEJoBEOUCOgUIABCRAjoCCC46CwguELEDEMcBEKMCOggILhDHARCjAjoECAAQQzoNCAAQsQMQgwEQyQMQQzoICAAQsQMQkQI6BwgAELEDEEM6BQgAELEDOgsILhCxAxDHARCvAToFCC4QsQM6CAguEMcBEK8BOggIABCxAxCDAToICAAQyQMQkQJQ7cs7WOekPGDNqTxoAXACeAKAAYoBiAG7JpIBBTI1LjI1mAEAoAEBqgEHZ3dzLXdperABBsgBCMABAQ&sclient=psy-ab&ved=0ahUKEwj3lazQkMXsAhVJZ80KHeWSAe8Q4dUDCAw&uact=5

I like the treatment in John Freund's Mathematical Statistics If a copy should still exist. Best, D. Booth

Best, D. Booth

I cannot fully explain this, but besides all similarities, there are some differences between correlation and regression.

1) as far as I understand it, the normality of errors (test: residuals) in regression and bivariate normality in correlation are needed for the inferential part and not the calculation of the parameters itself.

2) You have to keep in mind, that regression y ~ x does only represent half of the whole story, you can also calculate x ~ y. There parameter estimation will be different of course, but the inferential statistics for the predictor is the same. Therefore, the relationship x y has two possible regression slopes. And it is possible that in case of a linear relationship, where y is normally distributed but not x, the regression y ~ x may have normally distributed residuals, but not necessarily the regression x ~ y. So it seems important which variable is declared as x and y (which is typically derived from theory)

3) But this is not true for the correlation, both variables are assumed to be random variables (contrary to regression, where only y is seen as random variable, Miller & Miller, 2014). Another interpretation of the (Pearson) correlation is the geometric mean of the two regression slopes, hence it accounts for both slopes simultaniously.

These are my two cents, but I would love to see better and/or more in depth explanations from other contributors. ;-) THX

John D Crawford , adding a comment to the excellent answers of Rainer Duesing and David Eugene Booth , from my experience, in practice, it depends on why is one (or both) of the variables not normal: usually the case is the presence of outliers, which cause a bias in both the Pearson correlation and the estimate of the regression coefficient in a simple regression (one Y and one X). In this sense, I believe the assumptions are not contradictory, as the effect of the outlier(s) will translate very likely in non-normal residuals in the regression estimates.

Rainer Duesing As I understand the question, it is about inference and why the confidence intervals are not the same. That's why I gave the answer above. I may,of course, be mistaken. Rolando Gonzales As far as I understand your reply it is in error. Please note my reply above and that the x and y variable have different assumptions in bivariate regression. There is only need to discuss distributions in the inference cases and none with respect to the estimated Correlation coefficients alone. Hoping that I understood the question and with, Best wishes to all, David Booth PS PLEASE NOTE: Bivariate regression has 2 random variables while simple regression has only one. That is the difference I believe.

David Eugene Booth , I understood the question as one of point estimates and not inference... but I may be wrong! I changed my answer from "bivariate" to simple regression to keep us all on the same track.

The OLS regression, y ~ x, assumes the x values are measured without error, so y and x are not treated the same. You my look into total least squares regression (which has other names too, like orthogonal, Deming, principle component regression).

@Rolando bivariate means two random variables simple means one. They are not the same. Please read the reference.. D. Booth

John D Crawford, bear in mind that when one is working with real world data, normality is an approximation at best. George Box summarized things nicely as follows:

“…the statistician knows…that in nature there never was a normal distribution, there never was a straight line, yet with normal and linear assumptions, known to be false, he can often derive results which match, to a useful approximation, those found in the real world.” (JASA, 1976, Vol. 71, 791-799; emphasis added)

http://mkweb.bcgsc.ca/pointsofsignificance/img/Boxonmaths.pdf

To shorten Box's advice from Bruce Weaver , and mimic another quote from Box, Tukey (1986, Sunset Salvo from AmStat, p. 72) said: ``all assumptions are wrong'' \citep[p.~72]{Tukey1986}. Of course that doesn't mean that there are not degrees of wrongness nor that all assumptions are equal.

Short answer: No, not contradictory. As others have written, correlation: 2 random variables; regression: 1 random variable (DV), 1 or more fixed variables (IV). In regression, the residual is assumed Normal.

David Eugene Booth

as you already pointed out, the difference is in the inference for b and r (sorry, your google search links you sometimes provide, seem to show different results depending on the country, from where you try to access the link. I don’t think that you posted German sites, so I don’t know what you were exactly referring to). But isn’t there then a contradiction in the estimation of the statistical significance? I consulted the classic Cohen, Cohen, West, & Aiken – Applied Multiple Regression/Correlation Analysis…

Here we can find that for the inferential statistics:

t(b) = (b-H0)/se_b , where H0 is typically 0 and

b = r * sd_y/sd_x, where sd_y = sd_x = 1, in the standardized case and therefore

b = r

se_b = (sd_y/sd_x) * sqrt((1-r^2) / (n-2))

t(r) = r*sqrt(n-2)/sqrt(1-r^2), but since b = r, this is just an algebraic transformation of t(b)

So, t values for both parameters are identical (and hence the according p-values, since both have df = n-2). By contrast, confidence intervals do not agree (as David already mentioned). The CI for b will be calculated with the information above and the t-value for the according confidence level, so that the CI corresponds to the above mentioned results. On the other hand, the CI for r employs Fisher’s z-transformation, where CI limits are calculated for the z-value and then transformed back to r and which are not symmetrical around r.

The numerical differences may be negligible for the applied scientists (as I am), but I was wondering, which would be in general the more appropriate approach. In very rare cases, the p-value and CI for r would not agree (see screenshot). Does it mean that the t-value/p-value approach for r does not account for the different assumptions for r and b? Or am I missing something here?

Many thanks for any ideas!

Rainer Duesing Yes you are quite right the inference is different because the assumptions are different. The best approach depends on if x and y are essentially interchangeable.. In that case, the bivariate model would be preferred. Otherwise the univariate I believe. I think the univariate case is most often what we deal with but errors in variables regression does occur, especially in econometrics and some areas of natural science. Best wishes, David

Well, I don't worry too much about the assumptions of correlation, since I find regression more useful. So what stuck out to me in the question was this:

"...unless there is a very strong relationship between the independent and dependent variables (say X and Y, resp.) the distribution of the residuals is very close to that of the dependent variable, Y. (That is why the commonly stated assumption for regression is that the DV needs to be normally distributed.)"

Well, you want a strong relationship between x and y, and the DV does not need to be normally distributed.

I looked, but if anyone else picked up on this, I did not notice. If so, sorry: The unconditional distribution of y has nothing to do with the conditional distribution of y given x, assuming a model. I dealt with a great deal of establishment survey data which is highly skewed, and that does not cause the estimated residuals to be skewed. There is substantial heteroscedasticity, but that is a natural feature which is particularly noticeable in such a situation with a simple relationship between x and y, and when some x values are much different in size than others. (Both x and y were highly skewed.) You just have to use weighted least squares (WLS) regression rather than OLS - where homoscedasticity is just a special (and often unlikely) case of heteroscedasticity, where for homoscedasticity weights are equal. See https://www.researchgate.net/project/OLS-Regression-Should-Not-Be-a-Default-for-WLS-Regression.

The DV does not need to be normally distributed, or even close. That does not need to be a consideration at all. It is best if the estimated residuals (for WLS, the random factors of the estimated residuals) are close to randomly distributed, but that has nothing to do with the distribution of the DV. The question states that if the relationship between x and y is not very strong, then "...the distribution of the residuals is very close to that of the dependent variable, Y." I don't see that.

Daniel Wright noted that "The OLS regression, y ~ x, assumes the x values are measured without error, so y and x are not treated the same." I think this is important. But we have to be careful of lexical semantics. Errors-in-variables regression deals with measurement error for x-values, but that is not what "...measured without error..." means above. An estimated residual is only an "error" in the sense that the collected/observed value, y, differs from predicted y, because of random error and possibly including omitted variables or other model and population/data issues. But the error in x meant by errors-in-variables is just a measurement error which occurs in data collection but does not have to do with prediction.

When Daniel mentioned Deming, I'm not sure if he meant what I read in one of Deming's books (Deming, W.E. (1938, 1943, 1964), Statistical Adjustment of Data, 1964 corrected Dover republication of 1943 John Wiley & Sons, Inc. publication) on curve fitting and weighting of x and y, and x residuals, and y residuals, which confused me quite a lot. However, we definitely do "normally" think of y as something we might "predict" using regression, where x is the (or one of the) predictor/regressor/independent variable(s). Still, one might, as a colleague and friend, Joel Douglas, once pointed out, predict a few x values from yet another regression when collecting data, but that is another issue.