Dear Clement,

You have two broad assumtions in relation to outliers in a given data set. (1) Outliers can be real out-of-range values with exceptional and important biological (or other) meanings or, (2) they can be statistical artifacts of our study design, resulting from biases of our sampling methods or the statistical procedures applied. Before you take the decision of removing outliers for any transformation, you have to be aware of these two broad scenarios and act accordingly.

Now, if you are sure that the ouliers have no biological meaning, then removing them from the data set is "allowed", statistically speaking. But their removal might have an effect in your data set depending on (1) the number of outleirs, (2) the degree of skeewness they are giving to the frequency curve of your data set and, (3) their location within the data set (i. e; it would not be the same effect whether their are "one-tailed" or "two-tailed" located within the curve, in relation with their numbers).

I would recommend reading the Zar, J. (Biostatistical Analysis, Prentice Hall) book chapter on the matter first, and then move on to more advanced statistical texts such as those by Sokal & Rohlf.

.Cheers,

Hi Clement,

In software R you can check the influence of the outliers in your data. If there are outliers in your data is not because there is anyting wrong with them, but could be a  misspecification of the model. So, i dont know if you already check it, but it could help you. 

Best,

Lourianne

Dear Clement,

You have two broad assumtions in relation to outliers in a given data set. (1) Outliers can be real out-of-range values with exceptional and important biological (or other) meanings or, (2) they can be statistical artifacts of our study design, resulting from biases of our sampling methods or the statistical procedures applied. Before you take the decision of removing outliers for any transformation, you have to be aware of these two broad scenarios and act accordingly.

Now, if you are sure that the ouliers have no biological meaning, then removing them from the data set is "allowed", statistically speaking. But their removal might have an effect in your data set depending on (1) the number of outleirs, (2) the degree of skeewness they are giving to the frequency curve of your data set and, (3) their location within the data set (i. e; it would not be the same effect whether their are "one-tailed" or "two-tailed" located within the curve, in relation with their numbers).

I would recommend reading the Zar, J. (Biostatistical Analysis, Prentice Hall) book chapter on the matter first, and then move on to more advanced statistical texts such as those by Sokal & Rohlf.

.Cheers,

In the same line, outliers may be very important observations! Suppose that you are measuring the time series of natural mortality rates of a organism, annualy, for ten years.Then you find a massive mortality that exceds 10-folder the mean. Would you remove 10% of your information just to fit some given model at 0.05%? A combined matter of common sense, ethics, and ecological interpretation....

@Manuel: Thanks for this information but there is a shortcoming of the non parametric test of Kruskal-Wallis. Can other analysis like PCA etc be applied to the data after the non parametric test?

I would be very careful about the use of non-parametric statistics they are often misused and often don't solve problems with outliers and heterogeneity of variance. Data transformations can reduce the influence of outliers (log / square root etc.). I'd recommend taking a look at Zuur et al.'s 2010 paper (A protocol for data exploration to avoid common statistical problems. Meth. Ecol. Evol. 1: 3-14. DOI: 10.1111/j.2041-210X.2009.00001.x).

The (automatic) removal of "outliers" was responsible for the fact the the drastic thinning of the athmosperic ozone layer was initially not recognized. (for instance see Brown:Statistics for Environmental Engineers, CRC, ISBN 9781566705929)

Let us consider physically/physiologically impossible values as "true outliers" that have to be removed (it would be stupid not to remove values that are obviousely, undaubtly, result of an error). Those "strange values" that are left, possibly far away from the rest of the data, are then those outliers one has to deal with, somehow. In the following I am just talking about such outliers (possible values, but far away from the rest, without proven cause of the large deviation ["error"]).

Outliers are rare. By definition. If they are rare, they are rare, then they are not harmful. Either they average out (when a lot of data is available) or they will not distrub many results (when data is sparse). So actually one should not be concerned with outliers. Especially not when any frequetistic interpretation like p-values is aimed for.

When outliers are not rare, they are no outliers. By definition. Then something else is wrong. This can be the measurement technology / instrument, also the experimental design, or the model. A note to Willioam's post: transformations change the model. Thus, a large number of "appearant outliers" can indicate a wrong model, and changing the model makes these "appearant outliers" vanish.

@Jochen, thank you sir. Then I would ask, in a case of one or two outliers, should one ignore them and proceed with the statistics. Atimes, the presence of a singe or two outliers may skew a data set. And when the outliers are identified and removed, the data set becomes normal. What do you say to that? 

You may try to do some statistics to see what is the effect of outliers, as suggested above. Also, try to find the reason why there are some outliers, may be you made some mistakes. Or the outliers may mirror correctly performed experiments and your "good" results be the effects of some strong bias! Otherwise, I suggest that you do your statistics with and without outliers and make available all results to the reader. Put them in the appendix and discuss the results that you believe to be correct. The reader will decide what your finding is.

One or two outliers (well within physically reasonable ranges, but simply "far off" the rest) in what amount of data? If we talk about, say, 10 values, then we talk about 10-20% "outliers", what is clearly quite a lot. It is not unexpected that these outliers massively influence all results. There is no way to say what is better:  keeping them may give siriousely biased results, removing them may also bias the result. This information is not contained in the data, and thus you can not base a decision about keeping or removing such "outliers" only based on the data.

A possible indication comes from the frequency of the "outliers". As I said: if they are not rare, something more fundamental seems to be wrong. However, when there is only little data, even the judgement of what is "rare" is difficult. In the case of 2 outliers among 10 values, a rate of 3% is still not very incredible, and -depending on the technology and conduction of the experiment! - 3% might be considered "rare". But at the same time, rates up to 55% are not less credible, and this would be a clear indication of some major problem or a model misspecification.

The removal of an outlier will change the mean value by d/n where n is the sample size and d is the deviation of the outlier (in standard deviations). The variance of ths change is n^(-3). So consider you have n=20 and 1 outlier being 6 standard deviations away from the rest, then removing this will change the mean by 6/20 =  0.3 (in standard deviations). Maybe this is not much. If the sample size was n=100, the expected impact of removing such an outlier was only 6/100=0.06.

Regarding your second argument:

The fact that the remainding data looks "normal" is no good argument to remove outliers. The normal probability model may not be adequate, leaving important things unconsidered (just for instance: there might be a bimodal distribution, and this may be scientifically of utmost importance). There is a possibility that this is what your data tries to tell you (and the "outliers" are screaming the loudest).

Again, any suggestion for a further operation depends on the actual data (and how it is obtained, with all details), on the sample size, on the resonability of the model, and much more.

@Dragon and Jochen: Thank you sirs for the information. I really appreciate these invaluable inputs.

If you suspect the presence of outliers, I recommend transforming the data into normal quantiles (or also called Normal Scores). You will get a robust statistical analysis this way.

Manuel: I agree with you that the text by ZAR is an introductory level text that I use in a first course on biostatistics. It does not contain best practices by statistics experts. 

Do you want without loss of information, try to removed outliers places you can add with mean values and then compared with analysis of outlier removed data inference and mean value substituted data inference which is the best you can decide. 

Note: You confirm with more literature of this study area.

If I understood Manoj correctly, his friendly suggestion may take you on the wrong track. You must have really STRONG, objective reasons to remove some data! If you would then replace them with the mean value - this will be a clear fraud.

If you have to, please do what Raid suggested above.

Or do more, better controlled, experiments.

Not really; several Bayesian cluster algorthms are based on estimating values "better" by using neighboring values. The real problem will be hand picking which values are "outliers". This will lead to some serious errors.

Raid

If we know that they are "neighboring" values. But we a priori do not know where the mean value is!

Raid

Also, this all will depend on how the sample is big. If we have a sample of n=6 or 8, every manipulation of data will influence the results very strongly. And in this case we just cannot treat the outliers as the "missing values".

Outliers should be investigated carefully. Often they contain valuable information about the process under investigation or the data gathering and recording process. Before considering the possible elimination of these points from the data, one should try to understand why they appeared and whether it is likely similar values will continue to appear. Of course, outliers are often abnormal data points.