Say we have a dataset that has the following attributes:
- customer_id: There are a total of 1000 customers, each of them with a unique customer_id
- observation_date: The date on which we last observed the amount of water a certain customer has consumed (yyyy/mm/dd)
- amount_consumed: The amount of water a customer had consumed on the observation date (in liters, since the last observation date)
Now, as I said, there are a total of 1000 customers, all with observations taken at different intervals. The intervals are between 30~45 days and mostly unequal for each customer. The data spans over 2 years , and I have about 19~24 observation for each customer (19~24 observations per customer_id).
(I can procure data spanning over 14 years as well)
Now, I have a couple of questions:
1. From what I understand, We need to use Time Series Forecasting methods on this data. If I want to forecast the amount of water consumed by every customer in the next month, do I have to train a different Time Series model for every single user? (e.g. train 1000 different Time Series models, each belonging to a certain customer_id)
2. What do you think would be the best algorithm to achieve this? From what I gather, machine learning algorithms tend to perform poorly when we don't have a lot of data points. So what method would you suggest for this particular task?
3. Would the inclusion of monthly weather data (temperature, humidity, etc.) help improve my accuracy, since we don't have a lot of data points?
4. Could we use methods such as Matrix Factorization here?
Thanks in advance!
Yes, have to consider each customer separately and treat it as a dataset with date as the predictor and water consumption as target. You may need to sample the data by month or seasons and get additional data points. Use average monthly temperature over the 14 year period and any other variable you deem fit and regress
For a good understanding of your query, I suggest you the attached study and the links: Article Predicting Chlorine Residuals in Drinking Water: Second Order Model
Article Cytotoxic drugs in drinking water: A prediction and risk ass...
https://iwaponline.com/ws/article-abstract/19/8/2179/69623/Predicting-water-demand-a-review-of-the-methods?redirectedFrom=fulltext
Hello,
According to your description, it seems like a regression problem. It is also a time series problem. You can use the fbprophet for time series analysis. You may also try different regression techniques with hyperparameter tunings. Kaggle has some example notebooks for this type of tasks, those kernels might help you.
Thank you.
Shahir Asfahan, so basically we should have each user_id as our rows and each observation date as the columns, then train a time series model for each user to forecast the amount of water consumed in the next interval?
Kazi Amit Hasan do you suggest Machine Learning/Deep Learning models for such a task or do you think statistical methods will suffice?
Behnam Sajjadi , I am suggesting statistical logistic regression.
Statistical logistic regression : https://www.statisticssolutions.com/what-is-logistic-regression/
You also can try fbprophet. Link of that paper: https://facebook.github.io/prophet/docs/quick_start.html
- Badges
- Science topic