I've been using the random forest algorithm in R for regression analysis, I've conducted many experiments but in each one I got a small percentage of variance explained, the best result I got is 7% of variance explained, however the mean of the squared residuals is extremely small, which means that predicted values are close to the actual ones, I've plotted both in the same graph and they are almost identical, below you can see the plot of actual values (orange) and predicted values(blue).
Note that in the field of finance, it is so common to get low R squared but the result still considered reliable, I've seen this in many articles.
I'm just wondering if I can overlook the small variance explained and go on with the analyses since the predicted values are relatively accurate. If there are people who used the random forest in the financial analyses, I'd like to hear from them.
Thanks in advance
Hi, (soon-to-be-if-not-already) Dr. Med!
Here is a brief, open-access article which may be of interest to you. This paper addresses the issue of overfitting for several machine-learning algorithms, including random forest:
https://odajournal.com/2019/03/21/some-machine-learning-algorithms-find-relationships-between-variables-when-none-exist-cta-doesnt
You got to be careful with random forest. Its highly likely to over fit the model.
One way to find out if your results are good or not is to run a simulation. Take random train and test data build a model and see the variance and R2 for test data. Repeat this for different train and test ratios. If you arrive at similar result every time, you can proceed with your study. If not you have to go back to review your data.
The graph shown above is for the predicted values against the actual values, but since I'm using random forest for regression, I didn't split the data for testing sets and training sets, I think this is used for classification, what I did is jut to regress a dependent variable over set of explanatory variables, in addition I compared the predicted values by random forest to the ones predicted by linear regression, even though the linear regression has higher percentage of R squared, the predictions of random forest are a way better.
Even in regression, you have to split the data to test the models reliability. You can still calculate RMSE/MSE and R2 for the predictive vs actual results. That should help you understand the results and model better.
Dear Mr Nagdev
Splitting the data to test and training sets doesn't help my case, because it excludes a considerable amount of information, that's why I chose to use random forest, because unlike other machine learning techniques, it is supposedly protected against overfitting , and data split is not that common when using random forest, if I want to use data split, I'd rather use k-folds cross validation which enable you to use all data without losing any part of it.
Hi, (soon-to-be-if-not-already) Dr. Med!
Here is a brief, open-access article which may be of interest to you. This paper addresses the issue of overfitting for several machine-learning algorithms, including random forest:
https://odajournal.com/2019/03/21/some-machine-learning-algorithms-find-relationships-between-variables-when-none-exist-cta-doesnt
- Badges
- Science topic
- Similar topics
- Mathematical Sciences
- Graphs