I have recently been working on applying ML algorithms for time series prediction (forecasting demand of item). I am a bit confused somehow. So, my questions is as follows
1.Does the performance of these algorithms usually depends of the nature of the dataset i.e descriptive characteristics like mean, max, min etc. The machine learning algorithms i applied includes, Gaussian processes, backpropagation neural network and support vector machine.
2.I found Gaussian processes performs better than the rest but i am not too convinced with my results since the RMSE is still high for all algorithms i tried. Below is a summary for data i have.
item_1
Min = 100
Max = 40200
Mean = 11849
RMSE = 11756.5
I used WEKA forecasting package i have attached a time series plot
The quality of prediction depends on the relationship between the attributes and your label. Usually, some sort of feature engineering is required along with the use of some kernel, when there isn't a linear relationship.
Hi,
Yes, the performance depends on the nature of the loaded dataset. Since your result is not so good, you may need to consider the following action:
- clean your data up very well
- keep some data for evaluation, the WEKA tool allows you doing such action, and then obtain the results on this data
- perform attribute selection
HTH.
Dr. Samer Sarsam
Thank you all. As suggested for second part of my question, different performance metric and further preprocessing task can improve the results. I have tried to those suggestion. And from my results also the error decrease and increase as i forecast more steps ahead(i have attached a screenshot of this). Can i say this behaviour is because of the dataset itself (garbage in garbage out or) i.e my data is too noisy or.
No one can charactrize your data from the result you provide. If you prepared your data properly, where you considered detecting and removing outliers and extreme values, then no noise have to be in the data.
You need to try many algorithms and select the best output. You also have to make sure that the features you're using are good ones.
Cheers,
Dr. Samer Sarsam
- Badges
- Science topic