difference between outliers and extreme values. Are body should be removed from data when using prediction?
I would recommend the approach in the attached paper for dealing with outliers. These ideas can be extended to many other types of statistical models. Best wishes, David
Hello Muhammad -
I think when you compare "outlier" to "extreme value" you are getting at the fact that many people think that a value from the tail of a distribution is an outlier to be removed, but if you define an outlier as a point that does not belong ... is disqualified in some way ... then a value from the "extreme" tail of a distribution is just a potential outlier. You would expect that if you look at 95% prediction intervals for a series of independent predictions, that 5% of your data would be outside of those limits. Perhaps you should be using weighted least squares regression, or something more robust. Perhaps if you look at prediction intervals, you may find your potential outlier or outliers to be not so strange after all. They may really belong. Or perhaps now that the datum (data) you see as an outlier(s), or potential outlier(s), has (have) been exposed, you might check to see if such data may have been incorrectly collected. That would be called "data editing" which should not be done lightly.
(By the way, "extreme value statistics" is another topic, altogether.)
Cheers - Jim
- Badges
- Science topic
- Similar topics
- Methodology
- Research Methodology
- Research Papers