No, in linear regression, the dependent variable does not have to be normally distributed. However, one of the assumptions of linear regression is that the residuals (the differences between the observed values of the dependent variable and the predicted values) are normally distributed. This assumption can be checked by examining a histogram or a normal probability plot of the residuals.

It is important to note that even if the residuals are not normally distributed, linear regression can still provide useful information about the relationship between the independent and dependent variables. However, if the normality assumption is violated, it may be necessary to use alternative methods for statistical inference, such as bootstrapping or permutation tests.

Adding to what Lyudmil Antonov wrote, approximate normality of the errors is a sufficient (normality) condition for validity of the t-tests on coefficients and the F-test for the model, but it is not a necessary condition. The necessary normality condition is that the sampling distributions of the parameter estimates are approximately normal. See slides 9-11 in the attached PDF. HTH.

The answers from Lyudmil Antonov and Bruce Weaver state that the dependent variable does not need to be normal distributed. This statement is kind of wrong and could possibly be misleading.

Of cource is the assumption that the dependent variable is normal distributed - but with an expectation (mean) depending on the predictors:

Y(X) ~ N( µ(X), σ² )

This means that each obsered value yi is considered a realization from a random variable Yi ~ N( µi, σ² ). Each and every random variable (Y1, Y2,... Yn) is normal distributed, and all these normal distributions share the same variance but not neccesarily the same mean. Since each of these variables has a different mean, it's not easily possible to recorgnise the common distributional shape from looking at the observed data (yi) (unless all of them have the same predictor values so that µi = const.).

The regression model specifies the relationship between X and and the expectation of Y(µ) as some function of the predictors, and the fitted model provides estimates for µ, which I will denote with the latin letter M:

mi = f(xi) for the observed data, or M(X) = f(X) as random variables

When the model is unbiased, then the expectation of M(X) equals the expectation of Y(X). Hence,

Y(X) - M(X) ~ N( 0, σ² )

The difference Y(X) - M(X) is called the "error". The distribution of the error variable does not depend on X; it is normal with expectation 0 and variance σ². The observed realizations yi-mi = ri are called "residuals". They can be easily checked for their (empirical) distribution and if this fits to a normal distribution with constant (zero) expactation and variance (σ²). This is what the two mentioned above stated, but it follows from the fact that Y(X) ~ N( µ(X), σ² ).

I think this is important to understand because this is the very same for any assumed distribution of Y. For instance, if Y if binomial distributed, then Y(X) ~ Bink( µ(X) ), or if Y is Poisson distributed, Y(X) ~ Pois( µ(X) ). The "difference" to the normal distribution is that the distribution of the error here is not of the same family (e.g. normal, binomial or Poisson) as Y. That makes it more difficult to check if the distributional assumption makes sense using the residuals.

Jochen Wilhelm, I agree with everything you wrote from the 3rd paragraph to the end in your most recent post. However, I do not agree that it is potentially confusing to tell folks that the DV for an OLS model does not have to be normally distributed. I actually think far more confusion has been caused by failing to make that fact clear to learners. Also, as I said in my earlier post, I think it is very helpful to make clear the distinction between necessary and sufficient normality conditions for OLS models. YMMV--and apparently it does! ;-)

Thank you for your feedback, Bruce Weaver

I think the "simplistic" view that all yi are repeated realizations from one variable Y (with fixed mean) hinders to see or to understand the definition of a statistical model. It hinders to see the general principles of statistical models (other distributions, including random terms, including variance structures). Sure - it's simpler, and so it's preferably taught this way. But in most real scenarios the data-generation process cannot be seen as a producing repeated realization from a random variable. We learn that it makes no difference if we toss one die twice or two dice once. But in reality we are faced with the problem that there are no two identical dice - they are all loaded some way or the other, and we find that some loading systematically co-varies with other features that help us to better predict the outcome etc. This whole way of "statistical thinking" gets blocked (mentally) -to my opinion- by teaching the simple version.

I still think it's problematic in the longer run to teach

yi = bxi + ri with ri being a realization of R ~ N(0, σ²)

instead of

Y ~ N(bX, σ²)

I would be grateful if you'd show me what's wrong with that.

Hi Jochen Wilhelm. First, I believe that the students and other folks I deal with would find your first formulation easier to comprehend than your second formulation. I also think the first formulation is more likely to help them learn that they should not plot a histogram of the DV and be dismayed if they do not see a normal distribution. These opinions are based on my experience, of course, and I may be wrong. ;-)

Second, I do not understand what you see as "problematic in the longer run" with the first formulation. I don't think you are suggesting it is seriously flawed, are you? I gather, instead, that you are talking about subtleties that it fails to capture as well as the second formulation. If so, I think it can impair learning (in beginners) if we try to teach them all of the subtleties right off the bat. (But perhaps I just need to reread your earlier post.)

Cheers,

Bruce

Dear Bruce Weaver , thank you so much for sharing your valuable experience. I take it serious and will consider this in future. And yes, you are right, I am not suggestig that it is seriousely flawed. But I also don't think that this is some kind of extra-subtelity to teach. It is a different point-of-view, showing (to my opinion) more from the "statistical essence" rather than focussing on data (or on the observed residuals).

Not necessary. In population model, the distribution of the dependent variable depends on the distribution of the random item.

Thanks for the collegial and stimulating discussion, Jochen Wilhelm. I'm sure my "tortoise mind"* will continue working away on the points you have made. ;-)

* https://www.goodreads.com/book/show/599317.Hare_Brain_Tortoise_Mind

Mr. Ningbo Kou

I believe you know what is regression. Main purpose is to determine the mathematical relationship between dependent variable, y and independent variables, x, x may be more than one, using repeated values of y against corresponding values of x. If y is known to be normally distributed (as you highlighted) then there is no need to try for regression. You can form the equation using mean and variance. However, the regression analysis will tell you if the dependent variable, y is linearly or non-linearly distributed with equations which must be validated by r-square value and F-statistic. No need to think about distribution of y. Apply the technique and select the best fitted trend line.

I don't see a reason why the variables should be normally distributed. The most stable fitting is obtained with evenly distributed variables (nicely spread out, not clustered)!

If the dependent variable is not normaly distributed, then the error terms in the model are not normally distributed thereby violating the homoscedasicity assumption of the OLS.

Reasons why the dependent variable is not normaly distributed maybe because there is an outlier observation in the dataset, wrong model specification etc etc etc.

One of the remedial measure maybe to inspect your data for any outliers. If no signs of outliers, then try transforming the data into logarithms (if possible). Or better yet follow the steps used in elementary econometrics on how to remedy for "Heteroscedasticity"

But if you want to report the findings as they are, please attach a big red warning board with the writting "The author will not be liable if the results found in the paper are used for policy formulation"

Samuel Oluwaseun Adeyemo, I cannot find that Zhang (2016) on the Biometrical Journal website. Please check that you have cited it correctly, and if you have, please provide a direct link (if you can). Thanks.

Samuel Oluwaseun Adeyemo, did you by any chance use ChatGPT (or something similar) to generate your reply? I ask, because Daniel Wright speculated that you may have done that in this thread:

https://www.researchgate.net/post/How_can_I_estimat_Seemingly_Unrelated_Regression_sur_using_SPSS

Thank you for clarifying.

If you deal with establishment survey data, you will deal with highly skewed data. The dependent variable data are highly skewed, as are generally all distributions encountered. But the estimated residuals in the linear regressions may be close enough to normally distributed, though heteroscedastic.

I think one source of confusion though is that one has to be clear when you are referring to the (unconditional) distribution of the dependent variable, where normality does not matter, or referring to the conditional distribution of Y, given the predicted-y. That is the distribution of epsilon, which we'd like to be 'normal.' If you look at ANOVA, each data set should be "normally distributed" because they are analogous to the conditional Y, say Y|x, with x in bold type, for each case.

When we refer to the estimated variance of the prediction error, where we necessarily have e, not epsilon, that is broken into two parts, the variance due to the model and that due to sigma of e. The first part gets smaller with increased sample size, but you are always stuck with the second part.

Linear regression does not necessarily require the dependent variable to be normally distributed. The assumptions of linear regression refer to the residuals or errors (i.e., the differences between the predicted values and the actual values) being normally distributed, not the dependent variable itself.

In fact, it is common for the dependent variable in a linear regression to be non-normally distributed, especially when dealing with real-world data. However, when the dependent variable is strongly skewed or has extreme outliers, it may be appropriate to transform the variable using a logarithmic or other transformation to improve the normality of the residuals.

It is also possible that the EURO-D depression scale may not have a normal distribution in the population. However, this does not necessarily invalidate the use of linear regression as long as the assumptions of linearity, homoscedasticity, and independence of errors are met.

Therefore, it is not necessarily an issue with your handling of the SHARE survey data or your analysis. It is important to interpret the results of your linear regression carefully, and if there are concerns about the distribution of the residuals, it may be appropriate to consider alternative modeling approaches, such as generalized linear models or non-parametric regression.

The answer to the original question depends on what the questioner means by "linear regression".

The basic steps of linear regression involve fitting a linear predictor by least squares, doing a sum-of-squared residuals analysis, and using these to provide standard-errors for both parameter estimates and for prediction from the fitted predictor. None of this relies an any assumption of Normality, but only on the assumptions of homoscedasticity and uncorrelated-ness of the prediction errors. The estimators estimate what they estimate (which would not be what you estimate when fitting a model assuming some non-Normal distribution). The procedure is valid on a large-sample basis. It may well be enough for what you are actually trying to do.

So, why would you need to worry about non-Normality of the residuals? The answer is that you don't. The histogram of the residuals provides information about what sort of errors you would get from your least-squares-fitted predictor if applied in the future, which may well be enough for your purpose. If you still wanted to worry, a major effect of doing an analysis specifically allowing of non-Normal prediction errors is that you get better estimates of the parameters and hence better predictions from the fitted prediction formula. This improvement may be minor,

Other responses have gone on about validity of tests, but that only relates to the standard tests. If you are concerned with testing with possibly non-Normal errors then there are approaches to testing based on bootstrapping.

I believe others have mentioned that, before immediately attempting to treat observed non-Normality of the prediction errors, you should examine the overall suitability of the model structure being fitted. There are formal tests for things like homoscedasticity, but you would probably be best off going back to basics such as described in standard practical texts, that involve considering the effects of data-transformations and looking at helpful graphical displays of the data.

Other practical suggestions are: (1) split the dataset into two at random to assess uncertainty in estimates etc.; (2) deploy one of the "robust regression" techniques, again to assess any immediate conclusions.

David A. Jones I would like to add that non-normality of the residuals might be an indication that the used model (linear or whatever) is not the best possible model. Either there is a pattern in the residuals, in which case the model function needs to be changed, or the residuals are very scattered "heteroskedastic", which can indicate that other variables influence the outcome values.

Examples here:

https://www.lerenisplezant.be/teksten/FittingKVdm-Manual.pdf

I did a quick online search for the EURO-D scale and found some apparently high-powered papers, none of which provided any useful plots. I did find: "The scale yields a potential range from 0 to 12, with the number of depressive symptoms denoting the score." So apparently the dependent variable is a 13 point discrete distribution. It would be interesting to see if the histogram for the marginal distribution contains peaks similar to those in the plot of the residuals. One could consider the goal of a prediction analysis to be to reduce the uncertainty in the marginal distribution to that in the distribution of residuals.

Given the discrete nature of the dependent variable, and the sample-size here (apparently approx 50,000), some non-regression based graphical investigations would be worthwhile.

What type of regression you'll use depends a lot on the type of your dependent variable: interval-ratio, counts, or categorical (ordered or unordered categories). For interval-ratio data, better start with linear regression, check the residuals with diagnostic plots and if they are strongly patterned, try to include more independent variables or interactions in the model. For counts, use Poisson regression, and for categorical variables use logistic regression.

From your example above, it is some kind of scale, so I suppose your dependent variable is sums of item scores. These are counts, and therefore Poisson distributed, so better use Poisson regression. Linear regression is not wrong, but if you do it, better use log transform of the counts (to alleviate heteroscedasticity). This is the old way we dealt with counts before the advent of GLMs.

I am not familiar with the EURO-D, but this article confirms that it is indeed a count of items endorsed, ranging from 0 to 12.

https://www.zora.uzh.ch/id/eprint/213485/1/fpos-03-665004.pdf

EDIT. It could be treated as a proportion, of course.

In linear regression, the dependent variable does not have to be normally distributed. However, the model’s residuals do have to be normally distributed.