It is very easy for students to confuse the two because textbooks write an equation as, say, y = a + bx + u where u~N(0,sigma). The equation is estimated and we have ^s over the a, b, and u. The u-hats look like the 'u's and then to test if the distribution assumption is reasonable you learn residual tests (DW etc,) But the u-hats are merely y-a-bx (with hats over the a and b). We have no idea whether y=a+bx+u is the 'true' model. The idea that the u-hats are sample realizations of the us is misleading because we have no idea, in economics, what the 'true' model or data generation process. So we generally don't have a given model but we go through a model selection process. We include variables, then we drop some of them, we might change functional forms from levels to logs etc. etc. We end up using the residuals to choose the models (do they look uncorrelated, do they have a constant variance, etc.) But all along, we must remember that the residuals are just constructs of the data and the estimates of the parameters we put in front of those variables.

Could you name a particular misuse?

I'd say that "errors" and "residuals" can well be used interchangeably. However, "error term" is a term in a model, whereas "errors" or "residuals" are actually observerd differences between data and model prediction. Our expectation/knowledge about the errors is represented by the probability distribution assigned to the error term.

I asked this question in reaction to an issue raised by Verbeek on error term and residuals bearing totally different meaning. His suggestion caught my attention because I quite remember witnessing one Junior student use these words interchangeably during my service (as a teaching and research assistant 3 years ago) at the undergraduate level and was told not to do so. ....I am however very grateful for the wonderful opinion shared and the question raised from your side, Jochen...Your suggestion is well noted

Thanks a lot Jan for the wonderful contribution...Your point is well noted and very much appreciated

Oh ok, thanks a lot Balazs for the clarification. Your suggestion is well noted and very much appreciated

I agree with Mr Kotsoz that error is related to population and residuals to sample but tested through residuals. In other words residuals are estimates for errors.

thanks a lot Niaz for the opinion shared. Your point is well noted and much appreciated

The error term (also named random perturbation) is a theoretical, non observable random term responsible for the differences between the observed value for the dependent variable and its theoretical value according to the model. As the model parameters are unknown it is not possible to calculate the theoretical value nor the error term. What we can actualy do is to find the best estimators of the model parameters with some data (a sample), in the sample there will be differences between the observed values and those estimated by the model, those are the residuals. So, to clarify:

-Both error terms (random perturbations) and residuals are random variables.

-Error terms cannot be observed because the model parameters are unknown and it is not possible to compute the theoretical value.

-Residuals can be measured because the parameters can be estimated with a sample.

-Residuals can be taken as estimates of the error terms (that are "population" objects).

Impressive, thanks a lot Carlos for the wonderful opinion shared. Your suggestion(s) is well noted and very much appreciated

It is a common students' misconception, surprisingly also in the replies above, to think that residuals are sample realizations of errors. This is *NOT* true.

In the classical multiple regression framework

Y = X*Beta + eps

where X is the matrix of predictors and eps is the vector of the errors

the assumption on the errors is that they have variance-covariance matrix

V[eps] = sigma^2 * I

where I is the identity matrix. This implies that residuals (denoted with res) have variance-covariance matrix:

V[res] = sigma^2 * (I - H)

where H is the projection matrix X*(X'*X)^(-1)*X'.

Hence, even if the inspection of the residuals helps diagnosing the assumptions on the errors, residuals and errors are different quantities and should not be confused.

HTH

Simone

Interesting...thanks a lot Simone for the wonderful and brilliant response...Your point is well noted and very much appreciated

The error term is the random disturbance, that disturbs an otherwise stable relationship.

Residuals are the observed differences between predicted and observed values in our sample. I agree with Simone that residuals and errors are different, but we can nevertheless use the residuals as estimates for the errors. I will show the difference.

We have the linear regression model

Y = X*beta + er,

where er is the error term

Y is also the fitted value (=X*beta_est) + res (the residual), where beta_est ist the estimation of beta.

Therefore

res= Y-X*beta_est=X*beta + er - X*beta_est =X* (beta-beta_est) +er.

We see that res is not the same as the errors, but the difference between them does have an expected value of zero, because the expected value of beta_est equals beta if the necessary assumptions of the classical regression model are fulfilled.

Therefore we can use residuals to estimate the standard error of the regression model..

Interesting...Thanks a lot Horst for the wonderful response....Your point is well noted and much appreciated

Thanks a lot Roussel for the wonderful opinion shared. Your point is well noted

When I work univariate models fitting -using non linear predesigned equations- and apply the old squares method, I have observed that fitting may be good for the particular points but not for the interval in between each couple of data points. In other words, fitting is not good for the slopes of the curve. Also, if you work too many points the fitting improves as the exponent of the model increases, but the model curve may take sinusoidal shapes. I recommend to study first univariate samples fitting first and once we are sure of their error and residual analysis, then we may explain those terms to students and jump later into multivariate analysis. It is quite important that teachers understand fully the subject before expecting that students do it properly. Thanks, emilio

Thanks a lot Emilio for the point made..Your suggestion is well noted and much appreciated

I think much has been explained about the difference between error term and residuals. One can go all the clerifications. thanks

One can retrieve residuals from any regression or ‘fitting’ output; the difference between the actual and model predicted observation of the dependent variable or variables matrix. These residuals may be an estimate of the errors of the specification, but not always. A good insight might be had by considering decomposed error terms commonly encountered in frontier estimation. In my limited experience, getting the students to really look at the residuals and use them in model development is the more serious problem in applied econometrics.

Residuals are denoted with "u" and they represent the residuals of the population regression function, PRF. In PRF, you have population parameters, meaning, betas. and residuals.

In sampling theory, you take samples. By using a sample, by using OLS estimators, you estimate a regression function. This function is the sample regression function. In a SRF, you have parameter estimates meaning beta hats. By using a sample and your beta hats, you estimate the dependent variable, y hat. Now, you have the error terms. Residuals are for PRF's, error terms are for SRF's.

thanks a lot Edward and Ersin for the respective opinions shared. Your suggestions are well noted and much appreciated. I however need further clarification from Ersin on your point that residuals are for PRF's and error terms are for SRF's.

Great responses.

It is very easy for students to confuse the two because textbooks write an equation as, say, y = a + bx + u where u~N(0,sigma). The equation is estimated and we have ^s over the a, b, and u. The u-hats look like the 'u's and then to test if the distribution assumption is reasonable you learn residual tests (DW etc,) But the u-hats are merely y-a-bx (with hats over the a and b). We have no idea whether y=a+bx+u is the 'true' model. The idea that the u-hats are sample realizations of the us is misleading because we have no idea, in economics, what the 'true' model or data generation process. So we generally don't have a given model but we go through a model selection process. We include variables, then we drop some of them, we might change functional forms from levels to logs etc. etc. We end up using the residuals to choose the models (do they look uncorrelated, do they have a constant variance, etc.) But all along, we must remember that the residuals are just constructs of the data and the estimates of the parameters we put in front of those variables.

In my opinion, although the comments presented above have slightly different focuses, they are all correct and undoubtedly contribute to the understanding of the difference(s) between residuals and errors. However, the question, mentioned in many comments, is how to explain this difference to students better.

I will give one example from my practice. In the introductory course, I ask students to analyze residuals after (linear) regressions. They usually become surprised when they find zero correlations between residuals and all regressors. Most of them remember very well that CORR (X, er) MUST be 0, either they have BIG problems. So, they are very happy with this finding and think that their OLS estimators are OK (i.e., unbiased). However, when they find the same result after the 2nd, 3rd, 4th... regression, they gradually become anxious and ask me what is going on. I use this moment to explain the difference between errors and residuals.

Another example of that is to sum the residuals, since they add to zero in an OLS regression with a constant term. Residuals are constructs.

Thanks a lot John and Aleksey for the wonderful opinions shared. Your respective points are well noted and very much appreciated

On a related topic, residuals have a second rather naughty use in the practical world of economic forecasting and that is to compute 'add factors' to a forecast. It is fine that the theoretical error terms are i.i.d. etc. but equations go off track. In large macro models. there might be many equations that haven't been looked at in a while and, on revised data or over time, the model goes off track. The last six residuals might be +20, +18. +25. +19. +23. +27. The ideal solution is to go back to the drawing board but there isn't time and the practical forecaster would set the future residual, in this case, to say +20. There is one other issue with residuals and that is the difference between static and dynamic residuals. Consider the equation C = .06Y + .94C(-1) (basically the regression of real PCE on real PDI from 70 to 2013--I am not proposing this as a serious consumption function but to illustrate). The OLS residuals look small in 2013 (6, -9, -7 for Q1, Q2, Q3) but the dynamic residual obtained by substituting in each predicted value of C through the sample period accumulates to a massive -$203.bn in 2013. Because 1/(1 - lagged dependent variable) is 25 in this case, putting a static residual into the forecast will have its ultimate impact multiplied by 25 fold! Residuals in models with lagged dependent variables need extra special care!

Interesting...thanks a lot once again John for the wonderful illustration...Your point is well noted and very much appreciated

Simply, residuals are bases to estimates the error term time series, which is not available. it doesn't mean that they are always efficient to estimates the error term. It depends how the model is built well. If the residuals' characteristics admit the model's assumptions (like being white noise with a normal pdf) they can be used to build up the error term estimate; otherwise, the model should be modified by e.g. changing p in the AR(p) and/or q in MA(q) parts of an ARMA model or adding forgotten independent variables in an ARMAX model. The process of model modification should continue to achieve residuals with acceptable characteristics.

The terms RESIDUAL and ERROR, even what they represent the same thing, they are not exactly the same.

In regression analysis, each residual is calculated as the difference between the observed value and the prediction value, for different combinations of the levels of the effects included in the model. They are therefore particular realizations of the true errors, and are not real ones, just each of one is a particular estimate. To be more specific, the sum each of the squares of the residuals divided by the degrees of freedom for the residual, leads us to the Mean Square Error, which is turn an estimator of the variance of the response variable.

Yi= alpha + beta Xi + ui  (population regression function, PRF) and Yi = alpha^ +beta^ Xi +ei is the Sample Regression Function (SRF). In SRS alpha^ is the estimator (statistic) of  alpha (parameter) in PRF. ui is the random error term and ei is the residual. Most often people confuse and mix-up the two. We can draw a dividing line between the two. The main difference between ui and ei is that ui is not observable where as ei is observable as ei = Yi -Yi^. However, ei is used as a proxy for ui.

Yi= alpha + beta Xi + ui   (Population Regression Function).  ui is the random error term.

Yi = alpha^ +beta^ Xi +ei (Sample Regression Function). ei is the residual.

It follows:

ei = ui -  (alpha^ - alpha) -(beta^ - beta)Xi 

We see that ei is not the same as ui. The difference between them has only an expected value of Zero, if E[beta^] = beta and similarly for alpha^. For the unbiasedness of the estimators we need the zero conditional mean assumption E[u|X]=0.

Random Errors vs Systematic error 

Random Errors 

Random errors in experimental measurements are caused by unknown and unpredictable changes in the experiment. These changes may occur in the measuring instruments or in the environmental conditions.Examples of causes of random errors are:

electronic noise in the circuit of an electrical instrument,

irregular changes in the heat loss rate from a solar collector due to changes in the wind.

while

Systematic Errors

Systematic errors in experimental observations usually come from the measuring instruments. They may occur because:

there is something wrong with the instrument or its data handling system, or

because the instrument is wrongly used by the experimenter.

Two types of systematic error can occur with instruments having a linear response:

Offset or zero setting error in which the instrument does not read zero when the quantity to be measured is zero.

Multiplier or scale factor error in which the instrument consistently reads changes in the quantity to be measured greater or less than the actual changes.

Moreover, it might be added that the "error term" is usually a summand in an equation of an model or data-generating process. In contrast, the "errors" are unobserved realizations of a unknown data-generating process. 

https://en.wikipedia.org/wiki/Errors_and_residuals

interesting discussion.....what i understood is error is the disturbance in the original model and residuals are difference between the observed and predicted value of dependent variable a

that's correct.

Having a look at a Probit plot of the residuals is mostly telltale: if significant systematic contributions are missing from the model, it will show in that plot!