R² = coefficient of determination = 1-SS.res/SS.tot = SS.reg/SS.tot

(SS = "sum of squares", SS.res = SS of the residuals, SS.tot=total SS, SS.reg=SS of the regression="explained sum of Squares; Note: SS.tot = SS.reg+SS.res).

R² is the fraction of explained SS from the total SS. Since VAR = SS/n and the sample size (n) is the same for all (tot, reg, and res), R² can be interpreted as the proportion of variance explained by the regression. This means: Without knowing your predictors (just the response), this data has some variance, for instance say 2000. If you know can explain a part of the variation in the response by your predictors (i.e. by your regression), you will get an R² value between 0 and 1, for instance say 0.8. This indicates that the residuals of your regression have a variance that 80% smaller than the variance of the response: 2000*(1-0.8)=2000*0.2=400. Thus 80% of the variance of the response are explained by the regression, and 20% of this variance are left over in the residuals.

r = (Pearson's) coefficient of correlation = COV(x,y)/(s(x)s(y))

COV(x,y) is the covariance between x and y, s(x) and s(y) are the standard deviations of x and y, respectively. r is a measure of the strength of a linear dependency between two variables (x and y). In principle, it is a scaled (normalized) measure of the covariance.

Only in a simple linear regression with one (metric) predictor is r = sqrt(R²).

First, I would like to confirm that Coefficient of Determination (R-squared) is not at all the square of Correlation Coefficient (square of r). Had it been so, then R-squared could never have been negative. But, R-squared can also be negative, if and only if, the theory demands that Sample/Population Regression Function (SRF/PRF) should pass through the origin (i.e., without any intercept coefficient). Second, correlation explains deterministic relationship between variables, whereas regression explains stochastic relationship between variables. So straightaway, both cannot be compared for describing the precision of any relationship between variables. Correlation coefficient (r) explains the deterministic linear association between two or more variables, whereas Coefficient of Determination (R-squared) explains the variation in the regressand due to variation in regressor/s (i.e., explained variation). Though, R-squared describes the model fitness, but again two models should never be compared on the basis of the values of R-squared only.

Held: Correlation Coefficient (r) should never be used for precision of regression equation. Coefficient of Determination (R-squared) can partially be used for precision of regression equation along with other stylized facts, I think so................

Since you have a multiple regression, correlation coefficient is non-sense, because correlation only occurs between two variables, ether partial or not. But coefficient of determination can be used to determine the precision of the model, whatever the model is, no matter.

But be aware, if your independent variables are correlated, then it gets a little complicated.

R² = coefficient of determination = 1-SS.res/SS.tot = SS.reg/SS.tot

(SS = "sum of squares", SS.res = SS of the residuals, SS.tot=total SS, SS.reg=SS of the regression="explained sum of Squares; Note: SS.tot = SS.reg+SS.res).

R² is the fraction of explained SS from the total SS. Since VAR = SS/n and the sample size (n) is the same for all (tot, reg, and res), R² can be interpreted as the proportion of variance explained by the regression. This means: Without knowing your predictors (just the response), this data has some variance, for instance say 2000. If you know can explain a part of the variation in the response by your predictors (i.e. by your regression), you will get an R² value between 0 and 1, for instance say 0.8. This indicates that the residuals of your regression have a variance that 80% smaller than the variance of the response: 2000*(1-0.8)=2000*0.2=400. Thus 80% of the variance of the response are explained by the regression, and 20% of this variance are left over in the residuals.

r = (Pearson's) coefficient of correlation = COV(x,y)/(s(x)s(y))

COV(x,y) is the covariance between x and y, s(x) and s(y) are the standard deviations of x and y, respectively. r is a measure of the strength of a linear dependency between two variables (x and y). In principle, it is a scaled (normalized) measure of the covariance.

Only in a simple linear regression with one (metric) predictor is r = sqrt(R²).

Many thanks to Tapas Das, Ehsan Khedive, Jochen Wilhelm and Brian Altonen. I think I have understood the difference between the two expressions.

Take a look at https://www.researchgate.net/post/Should_we_report_correlation_r_results_or_the_coefficient_of_determination_r_square

Thank you for you reminding, Alba.

coefficient of determination is better because

R² = 1- SSe/SSt

SSe= Sum square of residual

SSt= Sum square of total

The R^2 is the squared correlation coefficient of the observed and the fitted values regardless of how many predictor variables you have. So they both provide the same result. The R^2 provides the proportion of the variability explained by the model you developed. Since you are squaring the correlation coefficient it can only take positive values.

The coefficient of determination, r 2, is useful because it gives the proportion of

the variance (fluctuation) of one variable that is predictable from the other variable.

It is a measure that allows us to determine how certain one can be in making

predictions from a certain model/graph.

The coefficient of determination is the ratio of the explained variation to the total

variation.

The coefficient of determination is such that 0 < r 2 < 1, and denotes the strength

of the linear association between x and y.

The coefficient of determination represents the percent of the data that is the closest

to the line of best fit. For example, if r = 0.922, then r 2 = 0.850, which means that

85% of the total variation in y can be explained by the linear relationship between x

and y (as described by the regression equation). The other 15% of the total variation

in y remains unexplained.

The coefficient of determination is a measure of how well the regression line

represents the data. If the regression line passes exactly through every point on the

scatter plot, it would be able to explain all of the variation. The further the line is

away from the points, the less it is able to explain.

I like Jochen's and Tapas' comments. But are there any papers specifically distinguishing these two measurement, in the way that they have explained them? I often see papers and textbooks trying to relate these two measurement in the context of linear model / relationship But they do not specifically point out that these are two different measurement, and do not emphasize that they are only related in linear relationship. It leaves an impression on the readers that R-squared is indeed the square of Pearson correlation by definition (a disregard to context). I think there are a lot of confusions on these two measurement, and it would be nice to convince people with a published paper / textbook. 

Jochen,

People frequently calculate squared correlation coefficient between observed and predicted values when models are nonlinear, solved by MLE, or mechanistic models.  Thoughts on this and any well known relationships that you are aware of?  

Poisson regression for example?

Seems like an intuitive measure, but it is not clear to me exactly what it means in each of these contexts.

I guess I'm looking for some free consulting!!

Dear John, as if I could consult you!

I don't have much knowledge about using R² in GLMs. To my understanding and opinion this does not add any value to the analysis. You have the log-likelihood, the deviance, and information measures like AIC, BIC and whatelse. R² is a measure associated to the residual variance, and it could possibly (but please don't ask me where!) make some sense to have a look at R² when the reduction of the residual variance by the model is a major aspect of the analysis. To my knowledge this may only be the case for "normal" LMs* and not so in GLMs where the variance is usually not independent of the mean.

There are many ways to specify a kind of a surrogate measure for R² in GLMs, which are called "pseudo-R²". A nice list can be found here:

http://www.ats.ucla.edu/stat/mult_pkg/faq/general/Psuedo_RSquareds.htm

So for GLMs there not one (pseudo!)R², there are plenty, and all have different properties, some nice, some ugly.

Another related source I found is a document about "R-Squared Measures for Count Data Regression Models - With Applications to Health Care Utilization" (but I guess you already know it):

http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.169.6693&rep=rep1&type=pdf

--

* see also the answer to the question "Is R² useful or dangerous" by whuber:

http://stats.stackexchange.com/questions/13314/is-r2-useful-or-dangerous/13317#13317

 Dear Ming

Coefficient of determination is widely used with regression equations. Statistically, the closer R2 is to 1, the better the fit.

Regards