I have seen MSE used to compare results of different methods of outlier detection. Also, in practice it may be tempting to eliminate more estimated high residual collected y values to make results appear better. But one should have good reason to label a potential outlier as being an actual outlier. (Note that some data could be actual outliers and still fall among legitimate data, falsely indicating a lower variance.) Further, heteroscedasticity in regression, which Ken Brewer showed should be expected, Brewer(2002), mid-page 111, will mean that throwing out collected y values with larger estimated residuals will tend to throw out y values collected with larger x-related values, which may easily not be outliers.
......
Brewer, K.R.W.(2002), Combined Survey Sampling Inference: Weighing Basu's Elephants, Arnold: London and Oxford University Press.
......
Remarks on this topic are solicited here.
Thank you. - Jim Knaub
In my opinion,MSE is not good as a sole measure of outlier detection because of these reasons. MSE may lead to misleading conclusion, increase in MSE may not always reflect the actual effectiveness, MSE may fail to capture detection performance and lastly,MSE does not reflect true model performance.
A little clarification as to what I meant: Just because a value is in the tail of a distribution does not mean it should not be there. Deleting extreme values which are legitimate will lower the MSE but this will be wrong. (In the case of regression, heteroscedasticity should not be ignored or it can make some values seem more extreme than they are in reality.)
Anyway, extreme values are not necessarily outliers. You can lower the MSE by dropping them, but you generally should not do that without some other reason to believe you are removing actual outliers.
A potential outlier is not always an actual outlier.
- Badges
- Science topic
- Similar topics
- Social Science
- Media
- Journalism