In addressing outliers in boxplot, some researchers have taken different stands:
1) extreme outliers - delete;
2) non-extreme outliers - re-check and if error, recheck boxplot. Otherwise, change the score to a less extreme value.
What are your views on the above solutions? Could there be any other alternative solutions ?
Tze, I don't know which "authors" you refer to, but to me these suggestions seem to be misleading/wrong.
How will you define an "outlier" if there are only 5 possible values? By design there can not be any impossible, not even implausible values!
The normal distribution is a probability model for continuous data, more explicitely for measurement errors. Likert data does not need to have any sensible center value to which meaningful residuals can be calculated. Techniques that treat Likert data as if it was continuous data from measurements (like t-test, anova, regressen and linear models in general) are simply the wrong tools. It seems to me that you are cought in these standard analyses, and you act like "if the only tool you have is a hammer, then everything looks like a nail"... (or, in my words: you seem to use a car to fly by driving fast enough over a cliff)
Stop thinking about these "standard procedures" and look for methods to analyze ordinal non-interval data.
Outliers may be the only data telling you the most interesting story!
If you have a well-defined signal-generation mechanism (factory, quality control, very standardized procedure) that generates a hell lot of data that has to be analyzed automatically, then a careful automated procedure of outliers is useful. What an outlier is must be defined considering the costs of false removals vs. the costs of false inclusions.
In academia, there is only one rule for taking out "outliers": if the value is physically / physiologically impossible. Actually, when I know that the measurement failed (shunt fault, known pipetting desaster, instrument failure, ,,,) then the values should anyway NOT be used for any analysis (there are no "outliers" in this sense, these are actually "missing values" [any recored value is anyway not trustable, even if it happens to fall within a seemingly reasonable range]).
Otherwise, I should think harder *why* some values seem not to fit to others. Wrong (error) model? Unconsidered predictors? Interactions? IMHO there is never a good reason to exclude outliers. As an emergency solution I would accept a comparison of the analysis with and without outliers.
I agree - you should delete data apriori because it is invalid, and not because it falls outside your desired range. Of course all data should be validated equally, not just outliers.
In my opinion what matters is not how far a data point is from the mean or from its predicted value. What matters is how much that one data-point influences the conclusions that will be drawn from the analysis.
The way you described your problem sounds like you're using a boxplot _instead_ of actual statistical analysis. If so, then that's the real problem and there's no point in even worrying about ouliers. A boxplot is just a way to visualize data. Plotting software generally lets you set the threshold at which a point is represented as an outlier.
Hopefully that's not the case and you are doing a statistical test. If so, you can use dfbetas-- you calculate the difference between the treatment groups on the full data, calculate the difference between the groups on data with one data point omitted, and subtract the second quantity from the first. Then, divide the result by the standard error of the second quantity. Repeat this for each data point. This is tedious to do by hand or in a spreadsheet, but R has a function that automates the process.
Data points with dfbetas greater than 2/sqrt(sample size) might be considered for omission, or more conservatively, examined more closely to see if there is something they all have in common that might explain their behavior.
I do accept that outliers could present an interesting insight. Perhaps, they could be used in my interview with key persons in the organization of my study since I am using mixed methods. However, they are presenting some challenges to the normality testing.
I was made to understand that I should have a normal data before I proceed with the more advanced statistical testing. Otherwise, the advanced testing may be invalid. SPSS normality testing revealed 0.00 in sig result (KS and SW) for all my questions in the variables and hence, the data was not normal (n=300). And I have also carried out non-parametric testing and result showed rejection of the null hypothesis. Hence, I had to look at the graphics aspect of normality test to determine data normality. i.e histogram, QQ-plot, Detrend QQ, boxplot, variance between mean and trimline mean and including human errors. So far, with the exception of 2 cases in boxplot, the rest i.e the histogram, QQplot, Detrend QQ and variance between mean and trimline mean, looked normal. And there were no human errors in the data entry. Based on this graphics perspective, is it acceptable for me to presume that the data is normal? Granted that I accept the outliers as part of the data and the normality of the data based on graphics interpreation instead of numerics (p>0.05) , could I still proceed with the more advance testing i.e ANOVA, Sobel test, sample t-test? Or are there any issues that I should be aware of?
Francis Bacon http://en.wikipedia.org/wiki/Francis_Bacon pointed out, we learn once when we establish the general rule, and learn again when we investigate the exceptions. Outliers may represent errors, either in recording the data or in observing the study protocol. But they may also represent the presence of a process that is absent in most of the data. In pregnancy, you will find high outliers for blood pressure because there are disorders of pregnancy that cause cardiac output to be abnormally high.
As for dealing with outliers, I think that you need to make sure that the substantive conclusions of your study do not depend upon the outliers. There are robust methods that attempt to deal with outliers by downweighting rather than deleting observations, with the aim of keeping the contribution of each observation to the model more or less constant. Unfortunately, many of these are resistant only to some types of outliers. More recent developments have produced methods that are more generally robust.
There is a very good discussion in
Verardi V, Croux C. Robust regression in Stata. The Stata Journal. 2009;9(3):439–53.
And they describe a suite of Stata commands for regression in the face of outliers.
Tze, I strongly believe that your data is Likert scaled. You should use analysis techniques for this kind of data and not try to somehow interpret this data as if it was interval or even ratio scaled. Note that "non-parametric methods" based on ranks are problematic because you are likely having a lot of ties.
Further note: Tests on normality are nonsense, since they are based on sifnificance and not on the relevance of deviations from normality. It is a long story, but to cut it short: If you have little data (say, n
Jochen, you are right. It is based on 5 point likert. For example, for not normal data, a number of authors had suggested to use histogram (are there any data points sitting in extreme positions) and QQPlot (a reasonably straight line suggests a normal distribution) to determine normal distribution. Since the histograms for my survey questions do not have any extreme points, I am considering of transforming the data if they do not follow the QQplot straight line.
I have been struggling to identify how far of a deviation is considered as suitable for data transformation. For example, could the attached diagram be deviating from the straight line? What is an acceptable reasoning of deviation? For example, 3 out of 5 points follow the line? Or could there be acceptable distance between a deviating point and the line ? If yes, what is the acceptance distance? Or should I proceed with my independent sample t- test, One-way ANOVA, Sobel test and regression analysis? Could there be any article that have similar issues?
Tze, I don't know which "authors" you refer to, but to me these suggestions seem to be misleading/wrong.
How will you define an "outlier" if there are only 5 possible values? By design there can not be any impossible, not even implausible values!
The normal distribution is a probability model for continuous data, more explicitely for measurement errors. Likert data does not need to have any sensible center value to which meaningful residuals can be calculated. Techniques that treat Likert data as if it was continuous data from measurements (like t-test, anova, regressen and linear models in general) are simply the wrong tools. It seems to me that you are cought in these standard analyses, and you act like "if the only tool you have is a hammer, then everything looks like a nail"... (or, in my words: you seem to use a car to fly by driving fast enough over a cliff)
Stop thinking about these "standard procedures" and look for methods to analyze ordinal non-interval data.
Hi Tze,
Jochen has already made some good points. To my experience Likert scales often produce "well-shaped" data when multiple items are combined into one score.
However, Likert scales as dependent variables are bounded to either side. With extreme responses (as opposed to centered), this typically results in highly skewed response distributions. This may look like a lot of outliers, but it is not. It simply is a feature of your data. For an illustration see the Wikipedia article on the beta distribution.
There is a pragmatic solution to this: rescale your response variable to the interval (0;1) by virtue of a logit transformation:
Yt = log( Yr/(1-Yr)),
where Yr is the rescaled response variable. This often effectively reduces the skew. The transformed variable has a range [-Inf, Inf], which is desired for linear models. Since you are dealing with Likert scales, it also is a safe transformation as it is strictly monotone.
There is only one issue: Yr may not include 0 and 1, as this results in -Inf or Inf. The more items you combine into one score, there more unlikely this is. If it still happens just add or subtract a very small constant.
As Jochen indicated, the proper way would be to use graded response logistic regression. Good luck!
Boxplot outliers may not be outliers for several reasons. For example
a) You are looking at data better represented as lognormal. Clues: the outliers are high values and a normal quantile plot is curved.
b) the outliers are only outliers because you are plotting the data as a whole. If the data are composed of different subgroups, the outliers may not be outliers when plotted against the distribution of their own subgroup
Outliers are extremely important. Bacon said that we learn once when we establish the general rule and we learn again when we examine the exceptions. Don't just throw away an educational opportunity!
Noteably, the chance of a normal distributed value being outside the usualy used whisker region is about 0.7%. Therefore, also perfectly normal distributed variables will show "outliers" in boxplots quite certainly when there are only enough values. For more than 100 values the odds are in favor of at least one "outlier" shown in a boxplot.
- Badges
- Science topic
- Similar topics
- Mathematical Sciences
- Statistics
- Statistical Software
- SPSS