It's no longer a surprise to realize a wide gap between advances in academic research and practicality in Industry. This discussion is about exploring this gap for a particular domain, which is Time-Series Forecast. The topic has had great many research advances in recent years, since researchers have identified promises offered by Deep Learning (DL) architectures for this domain. Thus, as evident in recent research gatherings, researchers are racing to perfect the DL architectures for taking over the time-series forecast problems. Nevertheless, the average industry practitioner remains reliant on traditional statistical methods for understandable reasons. Probably the biggest reason of all is the ease of interpretation (i.e. interpretability) offered by traditional methods, but many other reasons are valid as well, such as: ease of training, deployment, robustness, etc. The question is: If we were to reinvent a machine learning solution solely for industrial applicability, considering the current and future industry needs, then what attributes should this solution possess? Interpretability, Manipulability, Robustness, Self-maintainability, Inferability, online-updatability, something else?
From my personal experience, Deep Learning (DL) offers NO substantial improvement in time series forecasting accuracy vs. traditional methods, such as Holt-Winters, exponential and double exponential smoothing with or without trends, Holt-Winters with seasonality and trends, autoregressive integrated moving average (ARIMA), digital filtering, etc.
Regardless of the concrete technique (including deep learning and recursive neural networks in AI), all time series forecasting methods are based on the use of the historical past data to extrapolate them into the future. The fundamental assumption of any forecasting technique (implicit or explicit) is that time series represents a stable pattern that can be identified and then extended into the future. If a pattern of the past data-points is not statistically stable, then no meaningful future prediction (forecasting) is possible regardless of the sophistication of the forecasting technique (DL or not).
It seems reasonable to assume that too ‘old’ data-points do not practically affect (correlated to) the most recent data-points, let alone the future data. The data-points that are strongly correlated to the newer ones can be used for making the forecast. The data-points that are weakly correlated to the newer ones (or not correlated at all) should not be included for forecasting; otherwise, the forecast will likely be skewed. Thus, the use of a large number of the available past data-points worth of several years of observations is not needed for making a meaningful forecast; in fact, the large number of the past data-points can be detrimental to producing a reasonably accurate forecast regardless of the used technique (deep learning or not). The problem of the optimal number of the past data points (training set size) of a time series should be the primary focus for industrial applicability of the forecasting rather than a forecasting technique (DL or not).
hi.,
With Naïve Bayes Classifier algorithm, it is easier to predict class of the test data set. A good bet for multi class predictions as well. Though it requires conditional independence assumption, Naïve Bayes Classifier has presented good performance in various application domains.
Alexander Kolker Thanks for the reply. Your points are very valid and agreed upon in practice. Personally, my first approach to time-series modeling would be some version of ARIMA. Even when/if deep learning independently replaces traditional time-series methods, I'd still find it impractical in Industry. I can't just go to my directors and give them a forecast value, without explaining how it was reached and what significant factors impacted it, in order for them to establish trust in the forecast. Even I personally wouldn't have trust in my forecast in such case.
A good and wise manager would never make a critical decision based on a reason that sounds like this: "We fired up thousands of models on the cluster, using randomized sets of parameters, different feature settings, and layer architectures, then we picked the one that scored best on a holdout validation set after N iterations". If the manager has any bit of patience left after hearing this, then they'd ask "Why did that one have the best score? Why specifically its parameters/architecture work better? does it justify our massive investment in the cluster?", to which my answer would be "I don't have a clear idea, but I can explain to you how backpropagation works". At which stage he'd definitely run out of patience and say "You are fired"!
The point in the above hypothetical scenario is what I am trying to get to in this discussion, which is What do industry decision makers require from ML in order to confidently make better decisions? Perhaps industry leaders would be able to give better input here than academic researchers, or so I think based on my observations from academic researchers at AI conferences; they seem to be very distant from industrial reality (in time-series domain). Anyhow, my goal is not to be bound to any existent time-series modeling technique, but rather to reinvent an ML algorithm inspired by industry-use requirements. For example, I so far have identified the following requirements based on my industry experience:
- Models should be explainable/interpretable/relatable (e.g. ARIMA coefficients/P-values satisfy this criteria)
- Manager prefer to conduct probabilistic inference, examine multiple scenarios, best, worst and most-likely scenarios (e.g. Bayesian models satisfy this criteria)
- Statistical confidence intervals don't make much sense in industry, specifically with time-dependent error variance
- Models should be updatable/adaptive to changes in recent data, without having to frequently restart training and reinvest in data selection/preparation (e.g. neural networks support this criteria better with online learning)
- Support for very large multivariate datasets (deep learning seem to be more promising in this criteria than VARMA)
The above are some requirements I've identified, but I am hoping for reaffirmation and additional input.
C K Gomathy I doubt that Naive Bayes Classifier can be reliably applicable in time-series forecast
- Badges
- Science topic