Dear All
If in your dataset there are missing values as well as outliers, which one you treat first? Replace missing values (with mean, mod, liner interpolation etc..) or deal with the outliers?
I think dealing with outliers should be the first step. See if you can attribute any reason for their presence. If the overall distribution appears skewed then try some form of transformation including calculating logarithm of the response variable and then assess for normality of the transformed data.
What I would do in such case, first is experimenting with including all those existing outliers and also leave the missing values as it is (perhaps use any value that indicates it is missing e.g. -1). Then, it depends on your scenario/application, how extreme you want to exclude/include/subthose outliers. Also the way you are partitioning the data for training/evaluation in which some outliers/missing data could present in one set but not in the other.
It depends greatly on number, dimention, and variation of your data base. If you have large number of cases, little variation and 1 variable, go ahead and replace the missing value. Otherwise, you need to carefully consider the effect of missing value replacement.
Regards,
- Badges
- Science topic