It is generally agreed that data driven methods, e.g., ANN, ANFIS, requires large number of data if they are used to forecasting time series. However, I am not quite understand why. What are the consequences of data scarcity for time series forecasting when using ANN or other data driven methods. Is there any method to overcome this problem?
Hi Yang,
All these data driven methods you have named are based on Training- Testing mechanism. What this means is simple:You train your data on a given set of data, the model will get trained on the variability of the past data and then the derived models are applied on the future inputs.
These models do not have any understanding of the Physical Processes of the occurrences they are to predict. However, all of these data driven models drive a relationship between input and output data based on the training data we feed in.
Hence, the more data we feed in, my model gets trained on higher number of input - output combination, ultimately leading to better overall accuracy. (However even this is not always true -Consider a case where an occurrence of its kind has taken place for the first time in history, the model will have no relationship to model such future occurrence.) Its like studying for an algebra test, the more you practice, the better you perform because your mind gets trained on a number of algebraic problems. However you can still not get trained for an out of syllabus question right!!!
Hence, where we have scarce data, I make my models dynamic in sense that my model gets updated with every new entry in the data.Which means every time my models are derived on data till time t-1, applied on data till time t to predict value for t+1.
Another approach could be to use bootstrapping. This won't improve your prediction, but would incorporate a buffer band to counter uncertainty due to scarce data.
You can write back to me in case you you have any question on this.
Regards,
Vinit Sehgal.
Thank you for Sehgal's excellent response. Then I have a very simple question: what are the consequences when a data driven model is trained with scarce data? I guess the validation performance would be much worse in that case, but I am not quite sure if this is the case or if there is any other consequences.
Thanks!
Here is the bottom line.... prediction methods are naturally data driven, and designed to mine data. Therefore, scarce data will lead to less reliable and meaningless forecast.
I would echo Eliab's point that whether or not your models perform good on the validation data set, your forecast wouldn't be reliable in case of scarce data.
The main differences between the model-driven and the data-driven methods are the later work directly with the datasets (training, testing, cross-validation) to find the functional relationship between input vector and the output (forecast). The former does not need that, it is based on a sound theory (some physical law, etc). ANN, ANFIS, etc all learn by examples (data), the more the better.
In cases of scarce data you can still use these methods but you should use other training schemes such as Leave-One-Out (LOO) or n-fold cross-validation instead.
The more data you use to train will make your model better. The main consequence of the scarce data in ANN will be the deviation in the outcome/ poor performance.
Beside the points in my previous comment, I want to emphasis the point on scarce data the model loss its generalization on unseen data or new data. Data driven methods rely solely on examples (instances) to figure out input-output relationship, i.e., they learn by examples through the training phase. So more data leads to better generalization..
- Badges
- Science topic
- Similar topics
- Statistics
- Time Series