In Brewer, K.R.W.(2002), Combined Survey Sampling Inference: Weighing Basu's Elephants, Arnold: London and Oxford University Press, Ken Brewer proved not only that heteroscedasticity is the norm for business populations when using regression, but he also showed the range of values possible for the coefficient of heteroscedasticity. I discussed this in "Essential Heteroscedasticity," https://www.researchgate.net/publication/320853387_Essential_Heteroscedasticity, and further developed an explanation for the upper bound.
Then in an article in the Pakistan Journal of Statistics (PJS), "When Would Heteroscedasticity in Regression Occur, https://www.researchgate.net/publication/354854317_WHEN_WOULD_HETEROSCEDASTICITY_IN_REGRESSION_OCCUR, I discussed why this might sometimes not seem to be the case, but argued that homoscedastic regression was artificial, as can be seen from my abstract for that article. That article was cited by other authors in another article, an extraction of which was sent to me by ResearchGate, and it seemed to me to incorrectly say that I supported OLS regression. However, the abstract for that paper is available on ResearchGate, and it makes clear that they are pointing out problems with OLS regression.
Notice, from "Essential Heteroscedasticity" linked above, that a larger predicted-value as a size measure, where simply x will do for a ratio model as bx still gives the same relative sizes, means a larger sigma for the residuals, and thus we have the term "essential heteroscedasticity." This is important for finite population sampling.
So, weighted least squares (WLS) regression should generally be the case, not OLS regression. Thus OLS regression really is not "ordinary." The abstract for my PJS article supports this. (Generalized least squares (GLS) regression may even be needed, especially for time series applications.)
Heteroscedasticity is common in real-world data but violates OLS assumptions, requiring correction.
Shaban Juma Ally, that is why one should use weighted least squares (WLS) regression. When the coefficient of heteroscedasticity is zero - which should not happen - then WLS regression becomes OLS regression.
Well, yes. I haven’t looked all the associated papers, as I am retired and lazy, but let me ask some general questions on which you might like to follow-up.
(1) Since at least the 70s, standard text books have suggested graphical ways of investigating model deficiencies in regression scenarios. A question is how much heteroscedasticity can such approaches detect?
(2) Except in simple cases where a standard transformation solves the problem, it seems that any theoretical solution involves including various new quantities to represent the problem and one could either attempt a simple weighted regression or go for a fuller statistical model. In general terms, either approach involves introducing extra “unknown” parameters, and one needs consider the overall effect of the uncertainty in those extra parameters. In particular, if heteroscedasticity is weak and there is limited information about the size of such effects, using poor estimates of those effects may give a worse outcome than just omitting them, and such poor estimates may result from having limited data. One would also need to balance-in the purpose of a particular study ... whether it is looking for a direct regression-like predictor or a more general statistical understanding of the real world.
(3) Any statistical study would usually suggest where further research could be beneficial. Where heteroscedasticity is detected, collecting good new information would usually result in a data-set where a direct simple regression-like approach can’t be used. I guess that the next most similar model would be a mixed effects model (mixed random and fixed effects). There is an immediate question of whether that type of model, or any other more complicated model, would be acceptable or understandable within the context of whatever any practical study is being done.
The following are links so some recent work on practical ways of evaluating regression models, but they may not centre on the heteroscedasticity topic.
https://link.springer.com/book/10.1007/978-1-4612-1160-0
https://link.springer.com/book/9783031883644
Well David I think this is addressed by the "associated papers" above and Ken Brewer's book. I noted in my second paper mentioned that more complex models may sometimes not be heteroscedastic just because the predicted-yi are lacking in some way, so you may settle for a zero coefficient of heteroscedasticity, but when a simple model is appropriate, heteroscedasticity is generally more obvious. Yes, it may sometimes be difficult to know what to do but since, as explained in Brewer's book and expanded on in my first paper mentioned, heteroscedasticity in regression should occur when you don't have a common mean model, so using homoscedasticity as a default would then be the worst option.
Thanks for your interest.
PS - In using ratio models for Official Energy Statistics I found that heteroscedasticity was high, but I hedged it down to account for data quality issues among the smallest respondents. This was for establishment surveys.
The concept behind the statement that "Heteroscedasticity in regression should be expected" is that of a size measure. For those interested, perhaps you might look at sections 2 and 3 in "Essential Heteroscedasticity," https://www.researchgate.net/publication/320853387_Essential_Heteroscedasticity.
Thank you.
Some Methods for Measuring Heteroscedasticity, With Examples:
Alternative to the Iterated Reweighted Least Squares Method - Apparent Heteroscedasticity and Linear Regression Model Sampling:
https://www.researchgate.net/publication/263809034_Alternative_to_the_Iterated_Reweighted_Least_Squares_Method_-_Apparent_Heteroscedasticity_and_Linear_Regression_Model_Sampling
Estimating the Coefficient of Heteroscedasticity:
https://www.researchgate.net/publication/333642828_Estimating_the_Coefficient_of_Heteroscedasticity
https://www.researchgate.net/publication/333659087_Tool_for_estimating_coefficient_of_heteroscedasticityxlsx
Pakistan Journal of Statistics (PJS), "When Would Heteroscedasticity in Regression Occur," https://www.researchgate.net/publication/354854317_WHEN_WOULD_HETEROSCEDASTICITY_IN_REGRESSION_OCCUR
- Badges
- Science topic
- Similar topics
- Social Science
- Media
- Journalism