Given the input of the system is the request, its type, starting time, ending time and service, the output is which request may occur in next two weeks, its type and service. Two models are required, first for predicting the type of the request and a second for the service. Some suitable ML models are, Logistic Regression, Support Vector Machine, Neural Network etc.

you can use anomaly-detection-time-series-deep-learning, because of given the input and output of the system is the request, its type, starting time, ending time service

Anomaly in Time Series Data

AnomalyDetection R package -

It is a robust open source package used to find anomalies in the presence of seasonality and trend. This package is built on Generalised E-Test and uses Seasonal Hybrid ESD (S-H-ESD) algorithm. S-H-ESD is used to find both local and global anomalies. This package is also used to detect anomalies present in a vector of numerical variables. Is also provides better visualization such that the user can specify the direction of anomalies.

Principal Component Analysis -

It is a statistical technique used to reduce higher dimensional data into lower dimensional data without any loss of information. Therefore, this technique can be used for developing the model of anomaly detection. This technique is useful at that time of situation when sufficient samples are difficult to obtain. So, PCA is used in which model is trained using available features to obtain a normal class and the distance metrics are used to determine the anomalies.

Chisq Square distribution -

It is a kind of statistical distribution that constitutes 0 as a minimum value and no bound for the maximum value. The chisq square test is implemented for detecting outliers from univariate variables. It detects both the lowest and highest values due to the presence of outliers on both side of the data.

What are Breakouts in Time Series Data?

Breakout are significant changes observed in the time series data. It consists of two characteristics that are given below -

• Mean shift - It is defined as a sudden change in time series. For example the usage of CPU is increased from 35% to 70%. This is taken as a mean shift. It is added when the time series move from one steady state to another state.
• Ramp Up - It is defined as a sudden increase in the value of the metric from one steady state to another. It is a slow process as compared with the mean shift. It is a slow transition process from one stable state to another.

In Time series often more than one breakouts are observed.

How to detect Breakouts in Time Series Data?

In order to detect breakouts in time series Twitter has introduced a package known as BreakoutDetection package. It is an open source package for detecting breakouts at a fast speed. This package uses E-Divisive with Medians (EDM) algorithm to detect the divergence within the mean. It can also be used to detect the change in distribution within the time series.

Need of Machine Learning and Deep Learning in Time Series Data

Machine learning techniques are more effective as compared with the statistical techniques. This is because machine learning has two important features such as feature engineering and prediction. The feature engineering aspect is used to address the trend and seasonality issues of time series data. The issues of fitting the model to time series data can also be resolved by it.

Deep Learning is used to combine the feature extraction of time series with the non-linear autoregressive model for higher level prediction. It is used to extract the useful information from the features automatically without using any human effort or complex statistical techniques.

Anomaly Detection using Machine Learning

There are two most effective techniques of machine learning such as supervised and unsupervised learning.

Firstly, supervised learning is performed for training data points so that they can be classified into anomalous and non-anomalous data points. But, for supervised learning, there should be labeled anomalous data points.

Another approach for detecting anomaly is unsupervised learning. One can apply unsupervised learning to train CART so that prediction of next data points in the series could be made. To implement this, confidence interval or prediction error is made. Therefore, to detect anomalous data points Generalised ESD-Test is implemented to check which data points are present within or outside the confidence interval

The most common supervised learning algorithms are supervised neural networks, support vector machine learning, k-nearest neighbors, Bayesian networks and Decision trees.

In the case of k-nearest neighbors, the approximate distance between the data points is calculated and then the assignment of unlabeled data points is made according to the class of k-nearest neighbor.

On the other hand, Bayesian networks can encode the probabilistic relationships between the variables. This algorithm is mostly used with the combination of statistical techniques.

The most common unsupervised algorithms are self-organizing maps (SOM), K-means, C-means, expectation-maximization meta-algorithm (EM), adaptive resonance theory (ART), and one-class support vector machine.