I ran a hierachical linear regression on some data concering human values and opinion. The dataset is very high quality and has a large sample size ( it comes from the European Value Study). After I including an additonal variable, the F change was significant ( p < 0.01), but the R2 was extremely low at 0.00032. I understand that this variable should not be included in a regression model as adds almost nothing to the expianed variance, but what is the reason for that? On it's own this variable is siginficantly corellated with the dependent variable, but with inclusion of other variables it adds no predictive power.
The conditional association isn't exactly zero, that should not be a reason for including (or not including) the variable in the model. Why are you using statistical significance to make this decision? And what do those p-values mean anyway, particularly as the computer is probably calculating several of these.
I think that the new variable introduced is cointegrated with an already existing one given the fact it is highly correlated with it. That means that it was not necessary to introduce it into the model in the first place. That accounts for the low R squared value of the new model.
Daniel Wright All items have been included based on some theoretical assumptions and previous research of the subject. I've decided to manually enter the variables in steps because some of them are known to have a strong effect on the DV and I wanted to check if adding some other IV will in any way make the model better. As I mentioned in the question on of the added variable adds a very tiny amount of r2, far less the 1% of explained variance, so basing on that it should not be included in the final model. Obviously the F change was siginficant because the was some change. Why a variable that has a moderate correlation with the DV on its own ( measured with Pearson's r), but adds such little to the predcitive power of the model? Is it maybe because it is also higly correlated with other IV"s?
Bruno Kozakowski-Kędzierski , if your sample is big enough everything will be statistically significant. Information criteria (like AIC, BIC) will likely be of more use. With a large sample I assume you are also looking at non-linear relations and interactions. As far as having a moderate bivariate correlation, and then this dropping to near zero for the conditional association, this is related to collinearity. Procedures like the lasso can be useful with collinearity, but it is usually better to base what to include in your model on theoretical things. One thing that sometimes is useful is transforming variables. For example, a lot of spending may be related to income, so if you divide the components by income, this can help reduce collinearity (depending on the data).
I think you are on the right track here: " On its own this variable is significantly correlated with the dependent variable, but with inclusion of other variables it adds no predictive power." - " Why a variable that has a moderate correlation with the DV on its own ( measured with Pearson's r), but adds such little to the predcitive power of the model? Is it maybe because it is also higly correlated with other IV"s?"
You should investigate correlation before modelling and when modelling investigate confounding and interaction between independent variables. It is possible, that due to correlation, confounding or interaction, the "effect" of this high correlated variable is being altered through other independent variables in your model, resulting in a minimal contribuition to the overall explained variance (R2).
A better goodness of fit measure for the evaluation of the quality of your modell maybe an AIC instead of a R2 value (see comment from Daniel Wright): you should check it!, as well as the amount of residual deviance for each modell (eventually using ANOVA to compare the adjustment between models).
If this doesn´t bring you further, you should consider analysing you data with another model from the family of generalized linear models / non-linar models, depending on the behaviour of your dependent variable.
Alternatively you could try transforming your variables.
Good Luck!
Ana R. Valente dos Santos Cartaxo thank you for the insight !
Our advice is contained in the attached preprint:
Preprint Boosting and Lassoing Cancer Risk Factors: New Prostate Canc...
Please this applies to all linear stat. models. Best, D Booth
- Badges
- Science topic
- Similar topics
- Mathematics
- Statistics