I ran a multiple linear regression model which has one dependent variable and four independent variables influencing it. The R -square of the model was very high (reached 95%) but when I used the approximation for some cases, there was a significant difference in the calculated value compared to my research results.
According to the model, only one predictor has a very large impact on the output while the other three predictors are minimal. (This should not be the case, since they all affect the result)
Is there anything else I should consider when using regression? How can I improve the influence of the predictors? More data?
Howdy,
Not an unusual finding, though most people I know don't investigate their models as you did.
If the dependent measure is categorical ordinal (e.g., a Likert-type scale), here is a relevant paper:
- https://odajournal.com/2013/09/20/maximizing-the-accuracy-of-multiple-regression-models-using-unioda-regression-away-from-the-mean/
If the dependent measure is an interval scale, here is a relevant paper:
- https://odajournal.com/2016/09/19/novometric-analysis-with-ordered-class-variables-the-optimal-alternative-to-linear-regression-analysis/
Here is a paper which shows that regression-based analyses are adept at finding strong relationships in purely random data:
- https://odajournal.com/2019/03/21/some-machine-learning-algorithms-find-relationships-between-variables-when-none-exist-cta-doesnt/
Finally, the latter issue is not "cured" by using a larger sample:
- https://odajournal.com/2019/04/11/effect-of-sample-size-on-discovery-of-relationships-in-random-data-by-classification-algorithms/
Best wishes...
Some general suggestions:
Do remember that while a model may explain a high proportion of the overall variation in the data (e.g., 95%) it doesn't mean that it explains a high proportion of the variation for each case.
Collinearity issues might also exist; you may check the variance inflation factor.
Thanks a lot Paul Yarnold, Caleb A. Aldridge and Vasileios Papadopoulos for the answers. It seems for my case the regression model does not predict for the more extreme cases and works for cases around the mean. I do have to consider the variation for the cases and collinearity between the predictor variables in the model. Will work on it.
Figured out why one predictor dominant. Firstly the relationship between each predictor and the dependent variable is linear. When the value of the dominant is zero, the dependent's output is zero (this is like the origin in my data set and this is also true from my research results). At the origin, the other predictors have non-zero values. So the model is skewed towards this point and this enhances the impact of the dominant predictor while reducing the influence of the others.
So it looks the model does a decent job, but it is insufficient. Should I consider alternative modeling approaches?
- Badges
- Science topic