I am trying to predict peak demand using machine learning techniques. Current articles consider this as a time series prediction issue and consider a 7-day lag to predict peak demand. A ML model I am trying to apply considers new features for this prediction, and I applied it without a week prior value lag. I was challenged why I did not use lag values for time series prediction like this issue.
The objective of my project was to evaluate whether adding new features would improve the daily peak demand prediction and assess the effects of the new features. If I use new features to predict daily demand, should I also consider the previous seven days' lags as a new feature? Is it correct to combine several COVID-19 related features with the lag demand for peak demand prediction for an unstable situation like COVID-19?
Ps:
1- The model I used for prediction is LightGradient Boosting.
2- Data trained and tested during COVID-19 situation (2020 & 2021)
3- The weekly trends of my target value in 2020 and 2021 are as below figures.
Dear Negin Zarbakhsh
Lag characteristics are essentially goal values from earlier periods. For example, if you want to anticipate the sales of a retail shop in period t, you may utilize the preceding month's sales t1 as a feature. That is a lag of one, and you could argue it represents some form of momentum.
ARIMA models, like exponential smoothing, are among the most extensively used methodologies for time series forecasting. The term "AutoRegressive Integrated Moving Average" is an abbreviation for "AutoRegressive Integrated Moving Average." Forecasts in an AutoRegressive model correspond to a linear combination of the variable's historical values.
Due to the lack of previous data, judgmental forecasting is frequently the only accessible option for new product forecasting. When anticipating demand for a new product, the methodologies we've just discussed (Delphi, forecasting by analogy, and scenario forecasting) are all relevant.
Time series forecasting is the process of making scientific forecasts based on time stamped data from the past. It entails developing models based on previous data and utilizing them to make observations and drive future strategic decisions.
Kind Regards
Qamar Ul Islam
Qamar Ul Islam Thank you very much for providing such a detailed explanation.
I am applying my prediction model during the COVID-19, and the situation for the country is unsteady, so I'm not sure if the prediction could still be done with a lag. Lags are used in normal situations, as you explained. Let's imagine I use a 7-day lag, and at the same time, COVID-19 could change from one week to another. In that case, this lag feature would not be very accurate in all scenarios. Do you still believe in this case I have to add the lag feature as a new feature? Also, my new features provide me with an acceptable accuracy, which means this issue could be predicted by the new features. However, I have been asked to add lag features. The model I used is the LightGradient Boosting model.
P.s: I cannot use a one-day lag because in my case, weekend trends are different from weekday trends.
Dear Negin Zarbakhsh
I found these links useful, it may help you. Please have a look at them:
https://towardsdatascience.com/peek-to-peak-time-series-forecasting-to-predict-demand-for-meal-kits-53c055e77bfd
https://www.mdpi.com/2079-9292/10/20/2518/pdf
https://www.hindawi.com/journals/complexity/2020/6393805/
https://www.hindawi.com/journals/mpe/2021/9942410/
Also, try to train the LSTM model, because this model uses memory, it is explicitly designed to avoid the long-term dependency problem. Remembering information for long periods of time is practically their default behavior, not something they struggle to learn!
Pleasure Negin Zarbakhsh
Choose a high number of lags and calculate a penalized model (e.g. using LASSO, ridge or elastic net regularization). The penalization should reduce the influence of irrelevant delays, allowing the selection to be done more effectively. And Experiment with various lag combinations and either.
Fisher's Points. One of the most popular supervised feature selection approaches is the Fisher score. The method we'll employ returns the variables' rankings in decreasing order depending on the fisher's score. The variables can then be chosen based on the circumstances.
Kind Regards
Qamar Ul Islam
Xu Mei Lun Hi, the paper is under review and I will send it ASA I received the confirmation.
Xu Mei Lun: Our paper “Human Mobility-Based Features to Analyse the Impact of COVID-19 on Power System Operation of Ireland” is now available under the "Early Access" area on IEEE Xplore. The current version is freely available at:
https://ieeexplore.ieee.org/document/9724211
- Badges
- Science topic