I have been assigned the task of performing business sales forecasting using time series analysis. However, before starting the forecasting process, I need to identify and treat the outliers in the dataset.
To achieve this, I have decided to use Seasonal Trend Decomposition (STL) with LOESS.
I would appreciate your assistance in implementing this technique using Python or R programming language.
There are several methods for detecting and treating outliers in time series analysis. Here are some common ones:
It is important to note that the choice of outlier detection and treatment method should be based on the specific characteristics of the time series data and the research question being addressed.
You probably need to firm-up for yourself what you mean by "outliers". Other respondents have interpreted these as essentially being observation-errors that have no effect on other values in the observed series. But you might be wanting to deal with sudden isolated shocks that do then have some consequences for later data-values. This type of thing is included in modelling techniques that go under the label "intervention analysis".
You say that your main purpose is "forecasting". If you are looking for a general approach that can deal with forecasting in a situation where you have irregular observations due to outliers being discarded, than you might do well to look at stuff done under the label "Kalman Filtering" ... for this you have an underlying theoretical "process model" (supposed to reflect "reality") with a supplementary "observation model" in which observations for given time points may or may not exist or for which the variances of observation-errors change. The forecasting-component here arises because the "process-model" is treated as continuing into the future, with uncertainties in it having been initially guided by the latest observations.
So, there is a vast set of possible approaches. Your outline of possible approaches sounds good as a starting position, from which you can judge whether the more complicated theory would be worthwhile for the actual target of your investigations. Some good plots will help you with this.
David, can you provide the full links please, and not in cropped pictures? There is no need to reinvent the wheel. The Tramo-Seats procedure contains everything you need. It is available in several packages (gretl, X-13ARIMA_Seats, Matlab, ...). For an R user, I would recommend RJDemetra (https://jdemetra.github.io/rjdemetra/reference/tramoseats.html). It makes use of JDemetra+, a package developed in Java at the National Bank of Belgium and this can be better for a finer analysis. They are designed for seasonal adjustment but the Tramo part is aimed at automatic model building (and forecasting) in the presence of outliers. The treatment of outliers relies on scientific research started by Tsay (1988), Chang, Tiao, & Chen (1988), Chen & Liu (1993), Maravall & Peña (1992), Gómez & Maravall (2001) among others. In the past, Tramo-Seats worked only with quarterly and monthly data but this is no longer true.
- Badges
- Science topic
- Similar topics
- Methodology
- Research Methodology
- Research Papers