I have the following dataset:
SQ - SEX - Weight - letter - duration - trail - quantity
1 - male - 15KG - abc - Year 1 - 2 - quantity 1
- Year 2 - 3 - quantity 2
2 female - 17KG - cde - Year X - 4 - quantityx
- 16KG - Year Y - 6 - quantityy
- Year Z - 3 - quantityz
.... etc...
I want to make a prediction model that predict the quantity, but using classic machine learning models ( not deep learning ones, like LSTM or RNN ), i.e. linear regression, SVM , .. such that:
predict quantity of n individual at a certain duration ( duration A) what will be the quantity ?
n - male - 25KG - xlm - 34 - A - ?
What is the best was to treat and pre-process duration , trail and quantity features before fitting them to preserve their correlation with the target quantity ?
Dear Lubna Mahmoud Abu Zohair
I think the first thing you could do is remove the record number column.
After the variables that are categorical and only store two values you can convert them into Boolean values (1,0).
I suggest you read data mining books like Data Mining: Concepts and Techniques
by Jiawei Han.
Kind regards,
Cecilia-Irene Loeza-Mejía
Dear Lubna Mahmoud Abu Zohair
I suggest have a look to these links:
1. https://towardsdatascience.com/feature-handling-3f14c12ecbb8
2. https://towardsdatascience.com/machine-learning-how-to-do-the-feature-selection-2de56182cd9b
3. https://www.pluralsight.com/guides/preparing-data-feature-engineering-machine-learning
4. https://machinelearningmastery.com/feature-selection-with-real-and-categorical-data/
5. https://www.datacamp.com/community/tutorials/categorical-data
6. https://www.tensorflow.org/tutorials/structured_data/feature_columns
7. https://www.kaggle.com/general/16927
8. https://www.analyticsvidhya.com/blog/2021/04/how-to-handle-missing-values-of-categorical-variables/
Kind Regards
Qamar Ul Islam
Dear Cecilia, Qamar,
Thank you for your replies.
I think i didn't communicate my requirement very well.
Its not about the data pre-processing from A-Z.
As you can see in the given 2 examples, i have 2 people who each have multiple records, 2 or 3 durations , and multiple quantities. These quantities increased or decreased by time (duration). So, i am thinking of better way to re-arrange this set in a way that it can recall previous duration with its corresponding trail.
I thought of this in 2 ways, either to consider last duration & last quantity , or to have previous duration and previous quantities where previous here will take the average of all previously durations and quantities.
So from that re arranging perspective, any idea of reference that can show us sorts of re-arrangements that we can do in features, which help us taking previous facts in to consideration?
I hope that i am clear now. Looking forward for your replies.
- Badges
- Science topic