Dear temothy

residual analysis concern about the accuracy of the estimated regression line, as you know there is always an amount of error so we concern about  that error must be random, if it is not so we think about multicollinearity analysis, when the residual ditribution is not random (or not normal) you must think about high correlations between independent variables and try to fix it. The book of Pidhazur 1997 it will be helpful.

good luck

Rana

Timothy,

Your example does not clarify your question for me. I do not expect the dependent variable to assume integer values with normal residuals. And I wander why you are interested in conditional distribution of the independent variable given the value of the dependent variable.

Below is a short R code with simulation that might answer your question. Since y is a continuous variable, I have created a factor using cut function and then present conditional distribution of x given y is in an interval. Hope it helps.

Sincerely, Diana 

#

# simulation x non normal, residuals normal

#

# x - independent variable

x

Diana,

A very nice answer. Your histograms of both x and y show that they are not normally distributed. But in these histograms it seems that you have taken all x values (or all y values). That is not quite what I had in mind. I'll try asking a different way:

Are these statements equivalent?

1) The residuals are normally distributed.

2) For every value of x, the y values are normally distributed.

Maybe I look at plant growth as a function of nitrogen fertilization (from 2 kg/ha to 200 kg/ha). Because I see that the residuals are normally distributed, then I also know that if I select 32 kg nitrogen/ha (a fixed value) the response in plant growth will be normally distributed about the mean value through which my regression line (hopefully) passes.

Is that clearer?

Timothy,

This is perfectly clear, and correct. In linear regression (or analysis of variance, or t-test ... or any other general linear model) we model conditional distribution of y (dependent variable) for given value of x (independent variable) as normal distribution with a

1) constant variance (i.e. variance is independent of x ... that is homoscedasticity)

2) mean that depends on x in a way that is linear in coefficients (we can use a nonlinear function of x, but function must be linear in coefficients that need to be estimated)

This is different from the example you provided in the original question where you were looking at the conditional distribution of x.

We also assume that x is measured without error (or at least that variance of error in x is much smaller than variance of error in y). Ideally x values would be planned or "designed" to achieve the highest power in an experiment. We do it routinely in analysis of variance, but not so often in linear regression.

Now can the same thing be said for multivariate regression?

I have y=b1X1+b2X2+b3X1X2+e

Is it possible to have normally distributed residuals if X1 or X2, or both are not normally distributed for a fixed level of y? It is an unlikely occurrence, but is it possible that the non-normality in X1 and the non-normality in X2 effectively cancel each other to result in residuals that are normally distributed?

I wonder if this might be true:

If I have a large number of X each of which comes from a different distribution at a fixed y then I can say that as the number of X approaches infinity the distribution of residuals approaches normality. The different distributions can be anything (uniform, normal, beta, gamma) or have different parameters (normal(0,1) is different from normal(30,74)).

I like simple answers, so here goes. For single- and multiple-regression analysis, I prefer to transform both X and Y variables to have Gaussian ("normal") distributions, which is an extension of log-log regression equations that scientists often use. But I examine the distributions of X and Y variables, rather than worry about residuals.

 -Bob Vadas, Jr.

Dear Timothy,

'the residuals are normally distributed is equivalent to saying that the independent variables are normally distributed at any level of the dependent variable. This is not the same as saying that the independent variable must be normally distributed.'

speaks about two things;

1.'the residuals are normally distributed'; the (imho) most important assumption in regression analysis.

2.'normally distributed at any level of the dependent variable' is, somehow, about homoscedasticity. Actually it should be: the variance of the residuals  does not vary when the values of the variables vary. This is the second assumption for regression.

(there are more assumptions, like linearity)

About your main question:'Is it possible to have residuals that are normally distributed when the variables are not?'

The Central Limit Theorem states (in short) that the sum of many variables, regardles their distributions, has a normal distribution.

In fact the approximation is quite fair with 5 variables.

It is even better (but not necessary) when the variables, themselves are normally distributed.

The residuals are: (from y=mX+b+e): e=y-mX-b

The answer to your question is:

Yes it is possible to have normally distributed residuals even when dependent and independent variable are not normally distributed.

But it helps (the approximation is better) when dependent and independent variables do have a normal distribution.

Greetings, Pierre

Pierre,

#2 is not exactly what I had in mind. It is more, how can I interpret Robert's answer to be technically correct rather than just helpful as you put it in your last paragraph.

Attached is a figure from Neter, Wassermann, and Kuntner "Applied Linear Statistical Models". Lets say this is man-hours to harvest a plot and there are plots of different sizes. From 0 to y the number of man hours is not normally distributed. From 0 to X lot size is not normally distributed. For a fixed lot size, say 25, the distribution of man-hours is normally distributed. So the question was, can I get normally distributed residuals if the bell shaped curves in this figure were not bell shaped?

Central limit theorem: In biology, most effects do not have a single cause but are a sum (or product) of many factors. Sadly, I look at biological data and the general rule is that it is not normally distributed. How can this be if the Central Limit Theorem is true? One possibility is that there is another assumption in the Central Limit Theorem. A brief exercise with a random number generator and a computer shows clearly that the Central Limit Theorem applies only if all the factors are weighted equally. If I have biomass is a function of 20 variables, but the variable that accounts for 90% of the variability in biomass happens to have a highly skewed distribution, then the distribution of biomass will not be normal no matter what the other distributions are like. The Central Limit Theorem tells me that the overall outcome will be closer to Gaussian than the one highly skewed variable, but that doesn't really help much. That is the only way I can see to reconcile the statistical certainty with the biological reality.

Timothy,

about:

'to transform both X and Y variables to have Gaussian ("normal") distributions'

I have two objections:

-these tranformations do not always work.

-the interpretation of the outcomes of the analysis in terms of the input data is difficult and very prone to mis-interpretations.

about:

'examine the distributions of X and Y variables, rather than worry about residuals'

I must say that the assumptions for regression hardly say a thing about the variables; these assumptions are statements about the residuals.

Your question:'can I get normally distributed residuals if the bell shaped curves in this figure were not bell shaped'

The answer is: Yes.

about:

'I look at biological data and the general rule is that it is not normally distributed'

One reaseon is:

the data that you look at are a sample. Often these do have a t-distribution, for example.

Another reason is:

some data (count data, waiting times, growth processes etc) do not have a normal distribution because of the nature of the process.

But the parameters of the models may very well have a normal distribution.

Finally the CLT:"Central Limit Theorem applies only if all the factors are weighted equally."

Sure, of course. Adding some drops of wine to a bucket of water does not do the trick..

You have to taste the result; or test the assumptions.

Cheers,

Pierre

Timothy,

In the regression figure you attached the bell curves represent distribution of residuals, so it is not possible to have a bell shaped distribution of residuals if this curve is not bell shaped 8-). That is to say ... conditional distribution of the dependent variable (say Y) given fixed values of independent variables (say X1, X2, ...) is normal, where expectation is linearly dependent on the values of Xs (that is your regression formula), and variance is fixed and independent from Xs and Y (that is residual variance). 

When it comes to the central limit theorem, assumptions are that you have independent observations from a (one) distribution (with limited variance), and a large enough sample in order to have normal sampling distribution of the mean (or the sum). So ... if you sum variables that come from different distributions that are not independent you cannot blame the central limit theorem if the sum is not normal, disregarding the number of variables in the sum.

@Diana,

So requiring that the residuals are normally distributed is equivalent to saying that the variables are normally distributed (as long as one thinks of this as being conditional on Y)?

There was no blame on the Central Limit Theorem. It was more a qualification to protect myself. Without the qualification, the statement becomes false.

Timothy, you keep conditioning on Y (the dependent variable), however, linear regression is all about conditioning on Xs (independent variables). So ... conditional distribution of Y given Xs must be normal. That is equivalent to having normal residuals. Whatever happens the other way round is not of interest.

Values of Xs could be chosen on purpose in a way that ensures that we get the best estimates of regression parameters (e.g. in an experiment), or they may be from any distribution. In most situations it does not really matter.

My mistake, I should have said conditional on X, not Y.

So if I have Y=X1+X2+X1*X2+e, can I balance right skewed residuals from Y=X1 with left skewed residuals from Y=X2 to end up with normally distributed residuals in Y=X1+X2+X1*X2+e ?

I use PROC UNIVARIATE in SAS to make sure that statistical transformation of X and Y variables works; KISS principle. The amount of skewness gives a good indicator of what the transformation should be.