I am looking for an overview of methods to detect univariate contextual outliers in time series data. One example application is data from industrial plants in different (unknown) operation modes or slow trends in the time series, but no seasonal effects. Visually those outliers can be seen easily by a human.
In the attached graph visually the contextual outliers above and below the trend can be identified clearly.
Most global outlier detection methods can be used with an window-based approach. But a method, that automatically consideres the size of the context would be beneficial.
Are there any suggestions which methods are recommended for that purpose?
> "Visually those outliers can be seen easily by a human."
I can easily see outliers above the trend. But how to define outliers below the trend?
I agree not that clear. But right after data point 1500 i would see as an outlier, at the same time there is a trend in the data. I couldn't find research addressing the topic of finding an optimal window size.
My suggestion is to use so called Goutereau constant. This constant is invented to measure variations in time series. If you take subsamples, for example every second, every third, ... value and find differences you have anomalies. Simple. In any case, subsequent differences are way to detect anomalies in such cases.
Thanks about the idea of Goutereau constant, I haven't heard about it yet. Do you have any link to it or is it also known under another name? Thanks!
The easiest approach here is to use loess with robust fitting, see more here: https://rafalab.github.io/dsbook/smoothing.html
I also think that smoothing is suitable in this case. But still the window size remains as a tuning parameter. Adaptive smoothing looks promising (like double exponential smoothing).
Smoothing is regular procedure, but what about outliers !? Question is precise: how to detect outliers ? On link mentioned by Davydenko I found only one word "outlier". Subsequent diferences around outliers are not the same in case you use all data and every secondm for example. If smoothing procedure for such data gives different results, implication is obvious.
I think the idea would be to use the residual to the smoothed signal as an indication for an outlier, if the smoothing method is robust enough. Smoothing with adaptive window sizes like double exponential smoothing could work.
- Badges
- Science topic
- Similar topics
- Mathematics
- Statistics