I expecte that you can use Gohanson transformation (IN SPSS OR MINITAB ) of each variable distributed non normal distribution and after complete all transformations you can run OLS.

The normal probability plot of the residuals shows two populations of residuals. It can be fit with two straight lines. Plot the original data against normal probability. Does also have two populations?

There are a few points that call for attention here.

First of all: normality tests are not relevant with such a large sample size. One reason for that is the CLT, but there is also the problem that normality tests are very sensitive to sample size, and with 400000 observations they will test positive even for very small deviations from normality, that may not be relevant to your analysis. So, don't rely on normality tests too much.

That said, QQ normal plots are a good way to check for normality, and your data clearly is not normal. But what about outliers?

Here's my philosophy when it comes to outliers: no such thing as outliers.

I mean: outliers are "suspect" data point; they are so large/small that you suspect that they may be wrong. But you must check whether they are actually errors in data collection or management and should be excluded, or they are just true extreme values. Extreme values do exist and there is no scientific basis for their a priori exclusion from the analysis.

Also, it seems to me by looking at the QQ-plot that you are treating as "outliers" a lot of data points. Outliers are, by definition, a few influential values that may reflect errors in the data. You can't have a lot of outliers; if you have a lot of outliers they are not outliers, they are just outcomes of the data generating process. And it doesn't make much sense to take a distribution that is clearly not normal and looking for outliers assuming that the data is normal. If you take a Pareto distribution and look for values that are far from the median you'll find a lot of them, but they are not outliers, they are just values from the distribution. If you remove extreme values, well, most likely your distribution will have a much better approximation to normality, but that's not an acceptable method to achieve normality.

You should probably make your mind up: if you choose to trust the CLT and apply normal approximation, then it makes little sense to check for outliers and they should not bother you at all: the sample mean should still be normally distributed.

If, instead, you'd rather have a model that faithfully reflects the nature of the data so that your extreme values are not extreme anymore, then you should go for a GLM. Choosing one needs that you take in account what kinf of data it is, i.e., what is your outcome?

I expecte that you can use Gohanson transformation (IN SPSS OR MINITAB ) of each variable distributed non normal distribution and after complete all transformations you can run OLS.

If you are doing transformations because you want normality of the data, that is not necessary.  (Just be sure to 'fit' your data well if the relationship is curved (which can still be linear regression). Consider graphical residual analyses.)

If you are doing transformations to handle heteroscedasticity, it is more straightforward and more flexible to handle various levels of heteroscedasticity if you use an estimated or logically obtained default value for the coefficient of heteroscedasticity to determine the regression weight 'formula' to enter "w" into SAS PROC REG, for example, or however your software handles entering regression weights.

To obtain a value for the coefficient of heteroscedasticity when going from OLS to the more general WLS (weighted least squares) regression, you can use the following tool (and consider the references given) developed for various linear regressions:

https://www.researchgate.net/publication/333659087_Tool_for_estimating_coefficient_of_heteroscedasticityxlsx

I will add two things:

1. The central limit theorem is about statistical inference not curve fitting. This is a common misunderstanding of the CLT. By the way, the OLS assumptions are about inference(Gauss-Markov Theorem)

see: https://www.google.com/search?q=gauss+markov+theorem&rlz=1C1CHBF_enUS847US847&oq=gauss+markov&aqs=chrome.0.0j69i57j0l4.12732j0j8&sourceid=chrome&ie=UTF-8

On the CLT itself, For example see:

https://www.google.com/search?q=central+limit+theorem&rlz=1C1CHBF_enUS847US847&oq=central+limit&aqs=chrome.0.0j69i57j0l4.8573j0j8&sourceid=chrome&ie=UTF-8

2. I believe the best way to deal with outliers is to understand them when possible not to arbitrarily discard them. See attached.

Best wishes, David Booth

Alberto Ferrari Thanks for your answer! But, I'm a bit confused here! I added the QQ plot (attached), and it seems clear that the sample data is not normal. So, how should I proceed? Should I trust linear regression or not? I cannot consider those data points as outliers (as you mentioned as well), because I don't have a good reason why they are outliers!

Best

James R Knaub

-James

Thanks for your answer. If I assume that normality of response variable is not a serious problem, then what should I look into? In my case, the predictor is genotyping results coded as 0, 1 and 2 (homo-, hetero- and mutant) in an additive model, so I am not sure if the graphical examination of residuals against the predictor for homoscedasticity be helpful. So, can I concluded based on your points, that I can trust linear regression given the non-normality of response variable?

Best

Oveis

Here are some examples of linear regression with highly skewed data:

https://www.researchgate.net/publication/319914742_Quasi-Cutoff_Sampling_and_the_Classical_Ratio_Estimator_-_Application_to_Establishment_Surveys_for_Official_Statistics_at_the_US_Energy_Information_Administration_-_Historical_Development

But it sounds like you might be talking about some kind of count data. I'm not really familiar with it, but the approach is quite different. You might look into poisson regression to see if that does what you want.

The QQ plot of the residuals showed the possibility of two populations. The QQ plot of the data confirms the two populations.

Two straight lines fit the data in the QQ plot. You can form a two line regression to indicate the two populations. What is the explanation of the two populations? What are the properties of the data that could be the source of the two populations? This is not a statistics problem, but a physical problem. You have some good information about the system investigated.

Joseph L Alvarez Thanks for your response. In fact, these data are supposed to belong to only one population. Response variable is a biochemical measure (here, ALT concentration from individuals), and predictor is the genotyping for specific variant corresponding to those individuals. So, can you please tell me why this data is not simply "non-normal" and represents two-population background? Is there any test to indicate that?

Best

Oveis Jamialahmadi , what's your outcome?

I think with 400000 observations you should be able to apply linear regression even if normality is violated; but if you want to use a model that takes non normality into account (and possibly adjusts also for violations of homoscedasticity) then you should choose a model based on what type of outcome you have, and w don't know that.

Alberto Ferrari In my case, response variable is a biochemical measure (here, ALT concentrations taken from individuals (continuous variable)), and predictor is the genotyping for specific variant corresponding to those individuals (coded as 0, 1 and 2 to account for an additive genetic model). So, the scatter plot for predictor (x-xis) vs. response (y-axis) looks like the attached figure.

I find it hard to believe that a logarithmic transformation does not normalize the data. Check again. If that's actually the case, you may try gamma regression...

But I think you can safely apply linear regression here. If I was a reviewer I wouldn't challenge that.

Alberto,

I checked the nomality with ks test, Anderson-Darling test, and Jarque-Bera test, but all of them rejected H0 after log transformation. So, how can I test for normality in a large sample size like mine?

Let's say I am performing lots of regression analysis for different response variables. Is it reasonable to apply log transform whenever ks or other equivalent tests reject H0, even when these tests reject H0 of log transformed data?

May we have a look at the QQ normal plot of transformed data?

Both qq-plots and histograms for original and log-transformed data are shown. It seems that skewness is changing from positive to negative after transformation!

But why does this happen?

Apparently, it's not very skewed :)

I think you can safely apply simple linear regression on non-transformed data. Deviation from normallity is very low, and the CLT grants that inference is still valid anyway.

Just check for homoscedasticity; if variances are equal no further action is needed, if it's not, log-transformation should stabilize it.

Many thanks for your help and comments :)

The data plot QQ Plot of Sample Data versus Normal Normal has the ordinate labeled as Quantiles of Input Sample. The scale indicates negative quantiles. How was this calculated? What are negative quantiles?

I did not notice the ordinate scale when I answered.

You state, "Response variable is a biochemical measure (here, ALT concentration from individuals), and predictor is the genotyping for specific variant corresponding to those individuals."

How does "Response variable" relate to "input sample?"

How does coding for genotyping relate to what you are regressing?

Your original question concerned the residuals of a regression. What did you regress to obtain the residuals? ALT concentration (response) as a function of genotype?

Oveis -

I really don't know where this thread went, or what you really wanted. But I see that a comment I made four days ago may have been misconstrued, so I deleted it, and show it here so I can make a clarification, just to address my slopiness.

Here it is:

....................

Oveis -

Normality of your data is not a restriction at all. It is often desirable that the estimated residuals, or better, the random factors of the estimated residuals in weighted least squares regression (OLS being a special case), be 'normally' distributed, but I think the CLT does help there some with predictions. It seems that I may have heard it both ways regarding estimated residuals, but examining the predictions, i think you can see that the CLT should help with all parts but the estimated residual itself.

Perhaps you should concern yourself here with graphical residual analyses. That would help in various ways. You might want to look into it.

Cheers - Jim

..........

Regarding predictions and the CLT, I should have said that the estimated variances of the prediction errors consist of one or more parts from the model, and one due only to the estimated residuals. Many say the CLT helps with the estimated residuals, but I'll go with those who say it does not, because the standard deviation of the estimated residuals, like any standard deviation, cannot be reduced with increased sample size, a major requirement for the CLT, I'd think. But the part of the estimated variance of the prediction error due to the model is reduced with increased sample size.

Anyway, I'm not sure that any of this is relevant to your question, but thought I better provide this clarification anyway.

Thanks - Jim