"The map is not the territory."

I think the essential problem here is that you have the equation for linear regression wrong.

One way to express it is:

Yi = a + bXi + ei

where ei is the vector of residuals.

So in the case where Y and X are uncorrelated, b is 0, as you say, but Y equals e plus a, or e plus the mean of Y.

Another way to express linear regression is:

Y-hat = a + bX

where Y-hat is the predicted value of Y.

So in the case where Y and X are uncorrelated, b is 0, as you say, and Y-hat equals a for all Yi, and a equals the mean of Y.

You can see how this plays out in the following R code.

X = ( 1, 2, 3, 4, 5, 6, 7, 8, 9)

Y = (16, 9, 4, 1, 0, 1, 4, 9, 16)

They are, in fact, uncorrelated. (r = 0).

When Y is regressed on X, a = 6.667, the mean of Y, and b = 0. R-squared also = 0.

###### R code ######

X = c(1,2,3,4,5,6,7,8,9)

Y = (X-5)^2

plot(X, Y, pch=16)

cor.test(~ X + Y)

model = lm(Y ~ X)

summary(model)

residuals(model) + mean(Y)

The correlation coefficient only asks if the values of Y are related to the values of X. It is not a test that either variable is a constant.

The correlation coefficient is not zero. It is possible that it is not significant. However, a failure to reject the null hypothesis (the correlation coefficient is zero) is not sufficient to conclude that the correlation coefficient is actually zero.

X=c and Y=b

Both c and b have an underlying distribution with a mean and standard deviation (a 95% confidence interval).

As Salvatore states, the equations are missing the error term. So you have X=c + e and Y=b + e.

You can also look at regression analysis at the p-value for each coefficient. However, like with the correlation coefficient, a failure to reject the null hypothesis is not proof that the coefficient is actually zero.

Pearson correlation coefficient could reveal only the linear relationship, if it exists. If it is not nonlinear, or if there are no any relations between variables Pearson coefficient is not the tool to distinct these cases. The remedy could be of two types:

1) you could consider nonlinear model instead of linear; and in this case you have numerous opportunities because the form of the model in unknown. But, of course, ocassionally, you could come across the correct model.

2) you could use the correlation ratio instead of Pearson correlation coefficient to reveal the relationship. It's a bit complicated, but more sufficient method to reveal the relationship between variables.

I have understood the answer given by Vladislav Shchekoldin and I have got one answer to my question in the first paragraph of this answer.

The content in the answer given by Timothy A Ebert is not completely clear to me. I have found difficult to get proper answer to my question from this answer. Thanks to Timothy A Ebert for his attempt to answer my question.

My apologies I will try again.

Salvatore pointed out that the equations you provided are incorrect because they do not include the error term that is part of all linear regression equations.

I suggested that the constants in your regression equations are not actually constants. They are estimates of the intercept when the true value of the IV = 0.

1) As estimates there is error associated with the estimate.

2) It is unlikely in real data that the true value of the IV equals 0, and it is unlikely that there is no variability in this estimate: so the IV=0 and the 95% confidence interval is (0,0).

There may be a terminology problem.

Parameter: Y=mX+b is the general equation for a linear relationship between Y and X. In this equation m and b are parameters because there is no data, and I am not expecting any. Both m and b can take on any value.

Variable: Y=mX+b looks exactly the same as before I am interested in the relationship between the quantity of nitrogen applied per hectare and the yield of soybeans in kilograms. With my data I will estimate values for m and b and both are now variables.

Constants: These are invariant values within the system being studied. Something like pi or e are universal constants. If I write a computer program and at the beginning of the program I write Cnst=4.7, then Cnst is a constant so long as its value does not change for any complete run of the program. If there is any circumstance where the value can change, then Cnst becomes a variable. I am allowed to change the value of Cnst before I run the program, but Cnst is not allowed to change for any reason once the program is running if Cnst is to be a constant.

It might be better to think of a and c as estimates of the intercept when the IV equals zero. They are not and never will be constants in the mathematical or computer science sense of the term.

I have been able to extract one answer to my question from the answer just given by Timothy A Ebert in detailed explanation. Thanks to him very much for his endevour.

Your question assumes only "b" and "d" are regression coefficients which is wrong! The "a" and "c" parameters in your model are also regression coefficients! Thus, depending on the numbers of data points an behavior of the "y" vs "x" data set, you may need to assume a nonlinear regression model.

Very interesting query

I have been failed to understand how the parameters "a" and "c" can be regression coefficients as mentioned by Amir W. Al-Khafaji. More explanation on this point is required with proper Justification.

Y = b0 + b1X1 + e

Both b0 and b1 are regression coefficients. The first is an estimate of the y-intercept at X1=0. b1 is an estimate of the slope.

In the equation

Y = b0 + b1.X1 + e

b1 is no doubt a regression coefficient. It is the regression of Y on X1. But how is b0 a regression coefficient ? Is it regression coefficient of Y on X1 or other ?. It is true that b0 is an estimate of the y-intercept at X1=0 and b1 is an estimate of the slope of the line described / represented by the equation

Y = b0 + b1.X1 + e

Geometrically, b0 is the y-intercept and b1 the slope of the line described / represented by the equation

Y = b0 + b1.X1

b0 and b1 are both quantities that must be estimated based on the values that you have for Y and X.

Maybe think of b0 as more than just the y-intercept at X1=0. "b0" is a factor that shifts the entire line up or down. I can say Y=4X and I can tell you that the line will cross the Y axis at 2, but there is no way to graph these two pieces of information independently while keeping both true. If I plot Y=2 and then plot Y=4X I don't get Y=2+4X. Regression tries to find a relationship between the dependent variable (Y) and the independent variable (X1) and that relationship includes a constant and an error term. In a univariate model these are the three quantities that are estimated: intercept, slope, error.

If the linear regression equation of Y on X is expressed in the form

Y = a + bX

then "b" is the regression coefficient of Y on X.

But, I have not yet understood how "a" can be the regression coefficient of Y on X.

The above form of regression equation can be expressed in

Y - E(Y) = b { X - E(X) }

where E(X) & E(Y) denote the mathematical expectations of X & Y respectively.

From this form, it is clear that "b" is the regression coefficient of Y on X.

Now, if we compare the two forms of the regression equation then it is seen that

a = E(Y) - b E(X)

Thus "a" is a function of the "b"

i.e. "a" is a function of the regression coefficient of Y on X

but not the regression coefficient of Y on X.

You cannot do different things to each side of the equal sign without changing the equation.

Y=bX is not the same as Y=a+bX. You can write Y-a=bX, but that too will not reduce to Y=bX.

Y-E(Y)=b(X-E(X)) is not the same as Y=a+bX

Y=a+bX is not the same as Y=a + bX + e

In algebra we talk about a in the equation Y=a+bX as a constant. However, in statistics the value of a is unknown and must be estimated. There is uncertainty associated with the estimation. Thus "a" is not a constant, but is a variable to be estimated. I cannot estimate "b" without also estimating "a" and vice versa.

If the linear regression equation of Y on X is expressed in the form

Y = a + bX + e

where e is the residual / disturbing /error term,

then also "b" is the regression coefficient of Y on X.

In this case, the above form of regression equation can be expressed in

Y - E(Y) = b { X - E(X) }

(where E(X) & E(Y) denote the mathematical expectations of X & Y respectively)

under the assumption that "e" follows normal distribution with zero expectation i.e. E(e) = 0.

From this form, it is clear that "b" is the regression coefficient of Y on X.

In this case also,

a = E(Y) - b E(X)

Thus "a" is a function of the "b"

i.e. "a" is a function of the regression coefficient of Y on X

which means that "a" is not the regression coefficient of Y on X.

By the same logic I also have

b=(E(Y)-a)/E(X)

Thus, "b" is a function of "a"

i.e. "b" is a function of the regression coefficient of Y on X which means that "b" is not the regression coefficient.

You can't have it one way without the other.

Suppose, X and Y are two variables .

Then there are three possibilities:

(i) X may depend upon Y but Y does not depend upon X.

(ii) Y may depend upon X but X does not depend upon Y,

(iii) X depends upon Y and Y also depends upon X.

If Y depends upon X linearly then the linear dependence of Y on X is described by the regression equation

Y = a + bX

Here, b is the linear regression coefficient of Y on X.

This equation describes the linear dependence of Y on X but not the linear dependence of X on Y. Moreover, X may not depend upon Y in this case. Hence, the regression coefficient of X on Y is not existent here. Accordingly, "a" can not be linked with the regression coefficient of X on Y.

Has this become a discussion over terminology?

Looking quickly through the internet and a few text books, for the regression equation

Y = B0 + B1·x1 + B2·x2 + e

There appears to be agreement that B0, B1, and B2 can be referred to as "parameters".

I would say that the majority refer only to B1 and B2 as "coefficients", and mention B0 separately as the intercept. However, it's also pretty common to refer to all B0, B1, and B2 as "coefficients", for example specifying when they are talking about the "intercept coefficient".

Salvatore, The answer to your question is possibly but it is more than simply replacing "a" and "b" with B0 and B1 and including an error term. The error term is typically dropped in this conversation despite repeated suggestions that this is inappropriate. There may also be other problems, but I am starting to think that they are beyond my skill to help.

Dhritikesh, There is at least one more option in regression analysis.

iv) X is not dependent on Y. Y is not dependent on X. Both X and Y are dependent on an unmeasured variable Z, or a number of unmeasured variables Z1, Z2, Z3, and so forth. In all four cases you get Y=a+bX. In this equation one cannot distinguish between the four cases. Y=a+bX is the same as (Y-a)/b=X or if you prefer X=(Y-a)/b.

Regression is correlation, but correlation does not equate to causation. In correlation one can model Y=a+bX or X=a+bY. All this does is flip the X and Y axis in a graph of Y and X. The sums of squares, correlation coefficients, and p-values will be the same either way.

@ Salvatore S. Mangiafico,

I have understood from your answer that in the regression equation

Y = B0 + B1·X1 + B2·X2 + e

B0, B1 & B2 are coefficients.

However,

(i) B1 is the regression coefficient of Y on X1,

(ii) B2 is the regression coefficient of Y on X2

& (iii) B0 is the intercept coefficient.

My view is that B0 is neither the regression coefficient of Y on X1 nor the regression coefficient of Y on X2 .

This is what I mentioned in my question.

Dhritikesh, Sometimes looking at problems in a novel light is critical to making breakthroughs in science. In this case I think your view that B0 is not a regression coefficient is simply wrong and I see no way to change that.

@ Timothy A Ebert ,

I may be wrong. However, I have not yet found the logic behind it. I will have to be logically sure that I am wrong.

We have an expression Y=a + bX. It will obey all of the rules of algebra. You have mixed it with ideas that are not an inherent part of the mathematics. If X depends on Y, but Y does not depend on X then the equation Y=a+bX is wrong. Please keep in mind that in algebra Y=a+bX means that Y depends on X and X on Y. I can as easily write Y=a+bX as I can X=(Y-a)/b=a/b+X/b and these equations are equivalent. In statistics the interpretation is different. Y=a+bX means that X and Y are correlated. No causation is implied.

In the statistical expression Y=a+bX, "a" is not a constant but a probability distribution about some expected value. One could write Y=(a+ea)+(b+eb)X+e where the quantity (a+ea) cannot be separated (i.e. (b+eb)X does not equal bX+ebX). I suppose that one could break it down even further as (a+-sa+ea) where ea is error due to sampling or due to mistakes and sa is the variability from the underlying distribution.

One could think of "a" as the y-intercept when X=0, or as an offset that shifts the entire Y=bX equation some units up or down. Perhaps thinking of it as a constant is unfortunate because in this context it is not constant in the sense of being invariant. It also may change over time.