I think the combining phase should follow the model selection phase in TS data, as you can find that for a certain data set and a given metric (e.g. RMSE) the best model could be SARIMA while for another data set SARIMA is the worst model you can choose and you should not trying combining it with other models better performing. 

Here below you can find a link where 7 different models, i.e. 

• Autoregressive models

• Holt-Winters models

• Linear regression models with and without seasonal features

• Harmonic seasonal models

• Seasonal Autoregressive Integrated Moving Average models (SARIMA) and log-transformed SARIMA models

perform on 5 different data sets, i.e. 

• Sales of Australian wine

• Air Passengers

• Unemployment: Maine

• Supply of Australian Chocolate, Electricity and Beer

• Complaints to a motoring organisation

http://rpubs.com/Isaac/Comparing_time_series_forecasting_models

That said, once you had selected k top performing models among n available, I think a good combining criterion could be a posteriori model average. You can find how this technique works for instance in this book 

J. Friedman, T. Hastie, R. Tibshirani, The Elements of Statistical Learning, Springer, 2009    

The ensemble method is a very sophisticated option to combine the results of several forecasts models. A very simple option to start with is to simply use a weighted linear combination of the results that fulfills the following conditions for setting the weights:

There are a lot of publications out their in detail describing the approach mentioned above. See for example:

SANCHEZ, I. 2008. Adaptive combination of forecasts with application to wind energy. International Journal of Forecasting 24, 4, 679 – 693.

Another very interesting paper about ensemble regression can be found here:

Frank Rosenthal, Peter Benjamin Volk, Martin Hahmann, Dirk Habich, Wolfgang Lehner:

Drift-Aware Ensemble Regression. MLDM 2009: 221-235, https://wwwdb.inf.tu-dresden.de/misc/publications/RVH+09.pdf

read

http://www2.dbd.puc-rio.br/pergamum/biblioteca/php/mostrateses.php?open=1&arqtese=0721367_09_Indice.html

In general, I agree with the previous answers. However, there are some additions. A lot depends on the next. Do we know the real laws inherent in the dynamics of big data? If we know them, and they are stable, it is better to use the traditional forecasting methods. Otherwise it is desirable to use neural network prediction. In this case, the use of neural networks require no data models. They can be generated dynamically in the process of accumulation of neural networks experience.

Appreciate great answers. I see two approaches but I am not sure which one is better. I way be both of them or may be none.

The way I see the problem after reading these all great answers is that there are two different approaches :

1. An ex ante forecast approach but combine results of multiple forecast algorithms using MAP / ensemble.

2. An ex post forecast approach and pick the best performing forecast algorithms based on some evaluation criteria (splitting data into training and test set), use that to forecast.

3. One can hybrid (1 and 2).

There is a related problem suppose I pick a forecast model or take top N and use some weighted strategy to forecast. Whatever values my model forecast would be available in near future. So the question is once I have time-series data should one try to adapt and improve on the choice of my forecast model.

This may turn out to be an engineering problem, but in time-series how often forecast models get self-corrected once future time-series data point become available.

Hi Somil,

typically you are adapting your forecast model as soon as the actual data points are coming in and you can evaluate the quality of your forecast. The quality in most cases is assessed by using an error metric such as MSE, MAPE or SMAPE. Please have a look here for more information:

Another Look at Forecast-Accuracy Metrics for Intermittent Demand (article)

Author: Rob J. Hyndman

Journal: Foresight: The International Journal of Applied Forecasting

Year: 2006

Volume: 4

With such an accuracy metric in mind there are now several possibilities to decide when to re-estimate a forecast model. Typically, you would consider a sliding-window of N values where you are calculating the accuracy for. As soon as the error from one step to another deviates by X% (X greatly depends on your use-case, error acceptance, etc) you re-estimate the forecast model. You may further extend this by defining that a re-estimation is only done when the error threshold is violated K times. This enhancement safeguards you against single outliers.

When triggering the re-estimation you typically do the same as for the initial definition of your forecast model, you estimate the parameters using  a local (e.g., LBFGS, Nelder Mead, Gradient Descent) or global numerical optimization algorithm (e.g., Simulated Annealing, Generic Algorithms). As you see this is some kind of automated process now: Evaluation --> Adaptation Trigger --> Estimation --> Forecast --> Evaluation...

With respect to your first question, I would go for your first option or the third one (hybrid). The reason is that with estimating a forecast model using numerical optimization algorithm, you automatically gain the estimation error of the forecast model as a quality criterion. Using this quality criterion (estimation error) as an input for defining the weights in the ensemble, provides you with the best of both options. Very accurate forecast models are given a very high weight, while not so accurate ones only receive a small weight and thus, have only neglectable influence on the outcome. For sure you may also do a pre-filtering, by only considering forecast models that provide an estimation error below a certain threshold.

There is one more option and that is to use a multi-layer perceptron also known as a variant of an ANN. ANN have the advantage that you can define a back propagation algorithm that allows the ANN to adapt it self to new situations (i.e., new values). However, please be aware that defining a good working ANN is far from trivial. For your task you might decide between having the entire forecasting done by the ANN or only use the ANN for creating your ensemble.

Hi Colin, 

I don't know how much they are standard (actually, they're analized in the book by Paul S.P. Cowpertwait, Andrew V. Metcalfe, Introductory Time Series with R, Springer, 2009), but here you can find the links: 

• Unemployment: Maine:  http://elena.aut.ac.nz/~pcowpert/ts/Maine.dat
• Sales of Australian wine:  http://elena.aut.ac.nz/~pcowpert/ts/wine.dat
• Complaints to a motoring organisation : http://elena.aut.ac.nz/~pcowpert/ts/motororg.dat
• Supply of Australian Chocolate, Electricity and Beer:  http://elena.aut.ac.nz/~pcowpert/ts/cbe.dat

Best regards. 

I agree NN are notorious when it comes to modeling and it may require human intervention.

But what about scaling ? Do we have any methodology to scale the data which  makes forecast more accurate.  or it is a hit and trial method

On similar lines, I always see a problem where we try to forecast data based on month but instead bi-week split or quarter data points would have been a better choice.

Secondly what would be a good way to combine/split data-sets suppose if one gets daily feeds would be it accurate to take just add it up to make it month feed or if one has a quarterly data but needs to predict monthly forecast  take an average ?