Generally we deal with two types of forecasting; point and interval. With statistical method of forecasting (like ARIMA), we can perform interval prediction considering uncertainty of prediction. But what about doing the same with machine learning techniques like ANN and SVM?
In machine learning you search for (optimize) appropriate time intervals in batches using the "sliding windows" approach. Here, either data arrives in batches and/or forecasting is done in batches using some forecast horizon. Here, one or more batch or window size(s) are optimized. The regression/classification error is usually tested using 10-fold cross-validation, or preferably 10 times 10-fold CV. The one that tests the best is selected. There are a few papers on that: e.g. Klinkenberg, R. 2003. Predicting Phases in Business Cycles Under Concept Drift.
Some techniques allow that in regression (see for example Quantile Regression forests of Meinshausen).
In classification, the notion of uncertainty may be captured by a probability (probability that a customer buy this product for example). This is natively implemented in techniques such as Logistic regression or Random Forests.
interval predictions can be generated in many ways using statistcal models, in either frequentist or bayesian setting. A way to hedge predictions, i.e. to produce confidence intervals, that in my opinion is particularly interesting is conformal prediction. The theory of conformal prediction, which is based on frequentist statistics, can be applied to any model, even non statistics based ones. you cam find a tutorial about this here:
http://www.google.si/url?sa=t&rct=j&q=conformal%20prediction&source=web&cd=1&ved=0CC8QFjAA&url=http%3A%2F%2Fjmlr.csail.mit.edu%2Fpapers%2Fvolume9%2Fshafer08a%2Fshafer08a.pdf&ei=nH6-UND0BbON4gTG7oDwBA&usg=AFQjCNH08FikI3EDVytUdlC5H-545_qx_A
- Badges
- Science topic