Hi guys,
in the context of my master thesis i analyze the statistical relationship between income and subjective well-being (Panel: SOEP, n: 300.000 observations over 10 years).
After creating a model that is in harmony with the existing literature i conducted a fixed-effects "within" Regression (with robust standard error) that includes all relevant control variables.
I got a highly significant (0.01 level) regression coefficient of 0.1 for my income variable.
Despite that i received a R squared value of 0.06 and a negative adj. R. of -0.19.
I do not really know how to interpret the negative R squared. Does it mean my model doesnt fit and has no explanatory power?
I was expecting a small R squared duo to the many factors influencing subjective well-being, but not a negative one?
Anyone got an advice how to interpret this result? Can i still draw conclusions regarding the statistically significant coefficient and a causal link between income and SWB?
Im thankful for any advice!
Dear Gabriel!
I think your statement "I was expecting a negative R squared duo to the many factors influencing subjective well-being, but not a negative one?" is a typo, what did you really mean here? Did you expect a "small" R2?
I have some issues understanding your model, could you please elaborate more on it and show the whole model?
What do you mean with "within" regression (since you explicitly mentioned fixed effects)?
Which control variables did you include? How many? Why? The last question "why" is relevant, since it is NOT adviseable to include all covariables, only because you have measured them. Please look up "directed acyclic graphs" (DAG) and "collider bias". A reason why you shouln't include all variables, because this may result in spurious associations, which do not really exist between variables. This is especially true and needed, if you do not have experimental data AND want to draw causal conclusions, as you have stated yourself!!
You should really do not rely on statistical significance, with such a large sample size. Even the smallest effect will become statistically significant, but may not have any practical relevance. You should think about the size of the regression coefficient (after thinking about collider bias) and if this is relevant.
Your R2 is very low and a negative adjusted R2 is interpreted as zero. It is downweighted by model complexity, i.e. the amount of predictor variables in your model. What bothers me: Such a large divergence from the original R2 can only appear, if you have a (very!!!) large number of predictors, especially it it is true that you have 300,000 observations. Since adj. R2 is calcualted
1-((1-R2)*((n-1)/(n-p-1))), where n is the sample size and p the number of predictors.
This would mean you need to have more than 63,000 predictor variables to reduce R2 = 0.06 to -0.19.... so, there is really something odd in my opinion....
It would be nice if you could clarify the issues.
P.S.: did you account for the (assumed) repeated measures or did you just use a simple regression across all time points? Did you mean this with "within regression"? Again, it would b helpful to see the whole model.
Hey thanks for your answer!
This was a typo, i meant a small one.
My general model looks like this:
Yit = β1Xit + β2Zit + αi + uit
Y = life satisfaction
X = household income
Z = vector of control variables
α = indvidual effect ( Thats what i control for by using fixed effects, which means in the regression: model = "within")
With "within" i meant the "model" i worked with in the regression line. Its my first time working with panel data and i figured that thats the way to handle it in the regression and how to include fixed effects. This is how it looks in R:
spec1
A small addendum:
Until now i used an unbalanced panel, so that people who do not have observations for all years (2011-2021) are still included in my sample.
When i conduct the regression after balancing my panel (only individuals with observations for the entire observation period) the adj. R becomes less negative (from -0.19 to -.02), while the R squared is nearly the same (from 0.06 to 0.07).
In addition i conducted a regression only including one year (2021). As a result i did not use plm (with with model="within"). I used lm to conduct this simple cross-section regression. It turns out the adj. R now has a positive value of 0.15, which is fairly stable.
The problem then has to lay within my regression method based on plm (model="within"), right?
I cant find the problem since i just followed the wiki / textbook instructions on how to conduct a panel regression with fixed effects..
Now i found out, that when i use a balanced panel and increase the time frame i observe, i receive a positive adj. R. I increased the time frame from 2011-2021 to 1990-2021 and now have an adj. R of 0.07.
I might have an idea. Within my regression (plm, model=within) it does not compare individuals. The regression only looks at the variation of the income and SWB within the individual over the time, in order to control for individual fixed traits. Could the reason then be, that my time frame was too short and there was not enough variation within the individuals?
Sometimes it can be that easy!
I think i solved the problem.
It seems that the package (plm) im using has problems calcualting the adj. R when you use fixed effects (model=within).
I conducted my regression with alternative packages (e.g. fixet) and it then properly calculated the R squared (0.5) and R within which lays around 0.08.
Glad you solved the problem. I have no experience with plm, therefore, I would not be able to help further.
- Badges
- Science topic
- Similar topics
- Condensed Matter Physics
- Zero