Hi,
For my mastersthesis, I am doing an OLS-regression (sample size 75 with 4 independent variables). There seems to be a heteroskedasticity problem and I tried to fix this with robust std errors but afterwards my F-statistic was not significant, before it was.
Anyone tips to fix this?
You have to clarify your problem. What method did you apply to resolve? What pattern of heteroscedasticity did you encounter?
Hi,
Here is a reference for the resolution of Heteroskedasticity
https://datascienceplus.com/how-to-detect-heteroscedasticity-and-rectify-it/
This particular video might be helpful.
https://www.youtube.com/watch?v=uxoiJoLkdTw
Babak Jamshidi I detected heteroscedasticity with the breusch-pagan test. Afterwards I tried to run my regression with robust std errors but this made my F-statistic not significant.
When I remove a variable that was not significant out of the regression, the heteroscedasticity problem dissapears. Even more, the direction of the other variables remains exactly the same. Is this a way I can continue for my research?
If by removing that non-significant variable the problem is resolved, that is it; continue.
Sander -
OLS regression is a special case of WLS (weighted least squares) regression, where WLS regression uses the coefficient of heteroscedasticity, gamma, in the regression weights, to account for the heteroscedasticity. Sometimes people account for heteroscedasticity in a more ad hoc manner, but you can do better than that by considering gamma.
Heteroscedasticity in regression is a feature for finite populations, not a bug. Please consider the following:
https://www.researchgate.net/publication/320853387_Essential_Heteroscedasticity, as determined by Ken Brewer.
Here you will see how to determine a reasonable coefficient of heteroscedasticity, gamma, to use in your regression weights:
https://www.researchgate.net/publication/333642828_Estimating_the_Coefficient_of_Heteroscedasticity.
Here is a tool to use with your data:
https://www.researchgate.net/publication/333659087_Tool_for_estimating_coefficient_of_heteroscedasticityxlsx
Note information, on a sheet in the above linked spreadsheet, about default values for gamma. People often use gamma = 0, homoscedasticity, when they should not.
Here is a project, for further information:
https://www.researchgate.net/project/OLS-Regression-Should-Not-Be-a-Default-for-WLS-Regression, and the various updates in reverse chronological order there.
It sounds like you need WLS regression, where the regression weights are used in the model coefficients and are part of the error structure. The format will look the same as the OLS format. The estimated residuals can be factored into a random 'error,' and a nonrandom factor related to the regression weight, except that gamma is estimated with these estimated residuals with a tiny difference due to the difference between e and epsilon.
See my project noted above for more information.
You can read more on WLS regression in various textbooks. Online, the Pennsylvania State University online course links provide a great deal of useful information for anyone interested, and so far without a paywall.
Cheers - Jim
To see the best set of variables for predicted-y, a "graphical residual analysis" is helpful. That is also the first step in considering heteroscedasticity. To study whether you have fit too closely to this one sample to be a good model in general for your population or subpopulation, consider a "cross-validation."
Variables will show varying "significance" in different combinations. Also, a larger sample size will change the p-value. I suggest that you do not pay too much attention to 'p-values.'
Sander says "When I remove a variable that was not significant out of the regression, the heteroscedasticity problem dissapears." But heteroscedasticity is a natural feature to be expected. Sometimes it can be produced artificially by a model problem, but natural ("essential") heteroscedasticity is expected. Not having it could indicate a problem.
See "When Would Heteroscedasticity in Regression Occur?"
Preprint, June 2021, J. Knaub,
https://www.researchgate.net/publication/352134279_When_Would_Heteroscedasticity_in_Regression_Occur
To check your model, I suggest you research using a "graphical residual analysis." That will show model fit, including heteroscedasticity. To avoid fitting too closely to your sample and not so well to the rest of the population, I suggest you research "cross-validation."
Regarding a variable that is not "significant," I would not put too much emphasis on that. And also, the apparent relative usefulness of different predictors changes in different combinations. It is actually the predicted-y, not any single predictor, in multiple regression, which matters (and is considered a measure of size, by the way).
- Badges
- Science topic
- Similar topics
- Biological Science
- Histology
- Fixatives