Hello everyone,
I am calculating the duration from Site Selection to Clinical Start Date for the process map. The duration across all groups ranges from 0 to 70 months, with mean of 5.6 months and a median of 1 month.
I used the IQR score to identify potential outliers. Given the small size of the dataset, I am uncertain about which data points to exclude or include.
i found this number as a outliers: 22,22,22,24,28,72
Thanks
Dealing with outliers involves identifying them, determining if they're genuine or errors, and then either excluding, transforming, or adjusting them appropriately to prevent their undue influence on analysis or modeling.
The distribution seems to be highly right-skewed (mean >> median), so some extreme points on the right side are expected. Unless these data are not genuinely wrong or erroneous, they should not be removed.
The IQR is only covering the middle 50% of the observed values. I don't think that this is a good tool to identify outliers. Better you use the extension by "whiskers", as proposed by Cox' box-and-whisker plots. But these plots also only show values as individual dots that are outside of the "whisker range" to attract attention for further investigation - not to automatically classify them as outliers that ought to be removed.
As you said you have only a small data set, you can plot all the individual points to get a better impression of the distribution. You should also plot the logarithms, what is often more adequate for right.skewed data (like waiting times).
Ideally you have a distribution model (which might be some waiting-time model like Gamma or Weibull). Then you could judge how unlikely "more extreme data" would be, what gives you a hint of how (un-) expected such (or more extreme) values actually are. You can make a quantile-quantile plot and see if there are values being far off the straight line. This is much more instructive than just looking at right-skewed data and finding oneself alarmed by some "outlying large" values.
As Jochen rightly says, the data seem very asymmetric, however, I would like you to show the box plot.
The outliers can be classified according to frequency and remoteness.
Sometimes when the frequency is high, the existence of latent variables is possible.
But if you are interested in outliers, you may want to model it as well, for example with a Frechet.
Observing the data that you show, the one that calls deeply the attention is the 72, it is affecting too much the average and therefore the skewness and kurtosis, I imagine that these are very high values in the kurtosis.
Do traceability, detect associated variables that differentiate the measurement 72, maybe you missed some inclusion or exclusion criteria.
- Badges
- Science topic