I am working on a project relating to the field of speed prediction for street segments in future.
Imagine we got a dataset (which is not normally distributed) like this:
(Time-date | Speed | Density | Day of Week | Hour | Label)
(2018-12-27 12:30 AM | 78.5, 32 | 0 | 12 | 56.43)
It says one car with speed equal to 78.5 is driven on Saturday (`Day of Week` column) 2018-12-27 at 12 O'clock and its speed in another half an hour ('label' column ) is 56.43.
As you might have guessed the task is to predict the speed in another half an hour. The problem is that there is a very high correlation between speed and label and this makes issues for building a model.
When I get the prediction list and plot it against the test data labels I get `pic1.png`. The odd part for me is when I plot the prediction against speed, as you can see in `pic2.png`, the points are closer to the bisector which imply that model predicts the speed not the labels.
I did the normalizing, have used SVMregressor, NNregressor, Random Forest regression, Xgboost, .... but nothing works.!
It would be best if someone could suggest me a solution to deal with the problem with reasoning in details.
It's also appreciated if someone could send me some useful link.
Thank you guys
What is prediction list? Your model's speed predictions for test data?
Yes that's right. for each features row there is a prediction.
I would use a Generalized Linear Mixed Model (GLMM) and model Speed as a function of Density, Time-date, Day of Week and Hour, considering labels as a random factor. Using label as a random factor allows you to estimate a particular slope and intercept for each car. Of course you will need several measures for each car in order to fit the model.
Speed ~ Density + Time-date + Day of Week + Hour | 1+Label
If only two Speed values were collected for each car, I would calculate the difference (Speed at t2 - Speef at t1), and model the speed change.
DeltaSpeed ~ Density + Time-date + Day of Week + Hour
It could very well be that speed is by far the most predictive feature of your dataset; which isn't too far fetched as it is the same measure as your label - only shifted in time. If that is the case, other features will be "ignored" by your classifier (low weights, unused for splitting etc.).
Since speed is not a perfect predictor of label, you get a scattered plot of prediction versus label. Obviously, it is a good predictor of itself, hence your second plot.
Various approaches can be used to investigate this hypothesis.
I would personally start with a simple strategy: use a linear classifier (ordinary least-squares regression) with speed as the only feature and see how it compares with your results so far (using cross-validation or a test set depending on the amount of data available).
Other methods that can be relevant to check whether speed is the only feature to be consistently selected:
- use L1-penalized (lasso) regression, which can be seen as ODS regression with built-in feature selection. Note that your variables should be standardized.
- with random forest, compute feature importance ( "gini importance" or "mean decrease impurity").
- try to run various feature selection algorithms with various classifiers.
If it turns out I was right, it means that, as they are, most of your variables are not very useful for your regression problem. You could therefore try to investigate how they are distributed conditionally to your label in order to find a pattern which was not caught by your classifiers. Then you can try to transform or combine your features so that they can be used by your classifier.
can you give an example for combination method for features?
I'm a newbie in machine learning, don't know much !
Then, check the value of Time-date for outliers (specific times outside the cyclic week - hour routine - e.g. holidays).
Thinking about it, the issue is not the correlation between Speed and Label, which is a good thing because it means knowing Speed helps you to predict Label - precisely what you want - but rather the correlation between speed and the other variables. For instance, you expect the Speed to be lower for a higher density, and at specific time at specific days (e.g. 5 to 7pm from Monday to Friday). So in the end, once you know speed, the other variables may be of little help, hence what I mentioned in my previous answer.
Did you try to check whether speed was the only variable used by your model as I suggested? If this is the case, a strategy is to get more variables relevant to your problem, which are not related to speed.
Also,check Eduardo R. Cunha 's suggestion to model speed change. I would do the following: use Speed, Density, Day Of Week and Hour to predict DeltaSpeed.
Thus, your label predictions can be achieved in two steps: first predict DeltaSpeed, then predict Label as Speed + DeltaSpeed
Then, check the value of Time-date for outliers (specific times outside the cyclic week - hour routine - e.g. holidays, events).
Compare your prediction with a naive predictor.
It is possible that the algorithms simply learn to return the last value of a feature because they end in a local optimum.
I can recommend this article in this regard: https://towardsdatascience.com/how-not-to-use-machine-learning-for-time-series-forecasting-avoiding-the-pitfalls-19f9d7adf424
As a solution you could try to predict not the next speed but the change of speed.
- Badges
- Science topic
- Similar topics
- Analytical Chemistry
- Column