It is definitely interesting blog, as well as this one:

https://statisticsbyjim.com/regression/check-residual-plots-regression-analysis/

The basic assumption of regression model is normality of residual. If your residuals are not not normal then there may be problem with the model fit,stability and reliability.

In order to generalize a regression model beyond the sample, it is necessary to check some of the assumptions of regression residuals. If the graph of standardized residuals (ZRESID) plotted against standardized predicted values of the dependent variable (ZPRED) looks like a funnel or dots have a curved shaped pattern then it may be violating the assumptions of homogeneity or linearity. In case, if the dots seem to have a pattern and are more spread out at some points on the plot than others, then there could be violation of both homogeneity of variance and linearity. Adverse situations of any of these scenarios puts the validity of the model into a question.

Nevertheless, if dots looked like a random array of dots showing the normality and linearity of residuals, could be proceeded. Furthermore, if histograms looked like normally distributed and P-P plot looked like curving around a diagonal line it supports for generalizing the model beyond the sample.

non-normality of the residuals is an indication of an inadequate model. It means that the errors the model makes are not consistent across variables and observations (i.e. the errors are not random)

Siddharth -

Regarding prediction, normality of estimated residuals is nice in that it impacts the shape of the prediction intervals. If you graph endpoints of these prediction intervals on a plot with y on the y-axis and predicted y on the x-axis, you could get a nice picture of the situation, knowing intervals may be largely symmetric, though they would still be distorted to accommodate cases where negative values are not possible.

The estimated variance of the prediction error has a part from the model, and a part just from the estimated residuals. If predicting for finite population totals, then a larger sample size makes the model part of an estimate of the variance of the prediction error (for the total) smaller, but the residual part of the variance estimate does not get smaller with increased sample size, except that you are predicting for a smaller part of the finite population, so there is something like a finite population correction.

As for heteroscedasticity, that is a natural phenomenon which should not be ignored:

https://www.researchgate.net/project/OLS-Regression-Should-Not-Be-a-Default-for-WLS-Regression

It depends on what you are studying, but when you can look at some graphics, that is often helpful, and here you may see where producing graphics may sometimes be problematic.

Cheers - Jim

Siddharth -

I just looked at that Minitab blog for which you provided a link. I had assumed it erroneously made use of the Central Limit Theorem - which does not apply here as I explained about predictions for totals. What it does say is that with n>15 they had shown that nonnormality did not seem to hurt the usual statistical testing people do. (Since people often misuse and misinterpret those 'tests' by only looking at a p-value and using the same threshold for every case, and I find little use for them even if done well, I don't care for those tests much anyway, but it is good to know what they were talking about, though it was only for a specific set of test conditions.)

The blog by Jim Frost that Anton noted is generally very good. I might sometimes disagree to an extent, such as the way I look at heteroscedasticity, but from what I have seen of Jim Frost's "Statistics by Jim," I'd say it is a very good resource.

Best wishes - Jim Knaub

It depends. What is the purpose of the regression? How accurate do predictions need to be? When you plot the residuals do they correlate with one of the variables you are measuring?

You should probably explore reasonable steps like adding variables or complexity to improve the fit .

Since statistical inference depends on a sampling distribution. I would always check the assumptions in any regression I do. Best, D. Booth BTW why do you suppose robust regression was implemented, see attached.

Non-normality in the form of heavy tails in the residuals affects the power, sometimes a lot. See, for example, Rand Wilcox's textbooks on robust estimation.

Uniform and Normal distributions are just ideal cases, the problem is therefore how to cope with nonlinear multivariate regression models? In many situations the data may be positively or negatively skewed, also we should have concerns about kurtosis of the distribution. The remedy in such scenarios is rather to use parametric models, and saddle point approximation for calculating normalizing constants appears with Bayesian parametric modelling framework of maximum likelihood estimation.

Thank you everyone for your answers. Mr. James R Knaub could you explain a bit more about 'usual statistical testing people do', for example, if I want to see if social and emotional loneliness predicts depression among college students, where my sample size is above 15 but my predictor variables (social and emotional loneliness) are non-normally distributed than would a multiple regression be appropriate? What about if my outcome variable (depression) was non-normally distributed and predictor variables normally distributed with a sample size above 15, would linear regression still be appropriate?

Off-topic, when dealing with non-normally distrbuted outcome variables which method is better for multiple regression - bootstrapping or median regression?

You won't need any of those variables to be normally distributed for multiple linear regression. It is often desirable for the estimated residuals (the random factors of the estimated residuals in weighted least squares regression) to be 'normally' distributed, but there is no requirement like that for the variables for you to run your multiple linear regression.