From what I have read, OLS regression is the core of more complex modeling, such as used for panel data, with which I am not so familiar. Even in 'one-level' modeling, say the often used case of multiple linear regression, heteroscedasticity seems often ignored. More extremely, in cases of simple linear regression through the origin, heteroscedasticity is not always considered, though it has often been noted as a part of the error structure, and this natural phenomenon can often be shown to be very important. Weighted Least Squares (WLS) regression appears often to be brushed aside, or heteroscedasticity treated as a "problem" requiring a transformation, rather than left in the error structure where it naturally belongs. (One argument might be for hypothesis test use, but given misinterpretations of hypothesis tests, why not just stick with determining standard errors, and estimated variances of the prediction errors, using WLS?) So why, in more complex modeling does it appear that heteroscedasticity is often, if not always ignored? Is this just for simplicity, or has it been found to have less impact in complex models? Has this been studied? (Is heteroscedasticity considered implicitly, and I just did not recognize that when reading about random effects models?)
PS - I understand that there are variance differences between levels, but what about heteroscedasticity within levels?
Hi James --
Great question, and it actually gets at the topic of the master's thesis of one of the programmers I work with here at Penn State (Kelsey L. Ruckert, Improving the statistical method can raise the upper tail of sea-level projections). In the Bayesian hierarchical modeling we do, when heteroscedastic error estimates are available we prefer to incorporate these into our error structure because (1) Kelsey's work showed that the upper tails of the distributions of projected sea levels are quite affected by neglecting this aspect of the uncertainty and (2) to throw them away or average them out to a homoscedastic error estimate throws away useful information.
These are fairly complex models, but the Bayesian framework is simple enough to incorporate the more detailed error structure. With a long time series and/or complex error structure, the heteroscedastic error model becomes a bigger problem, leading to larger matrices that represent the uncertainties. And the more detailed your error structure, the more dense this matrix. Then you need to invert it... potentially thousands or millions of times, which can be a slow process.
An overview of a really simple example of the heteroscedastic error in the statistical calibration of a model (ours is an Antarctic ice sheet model; you could easily use a simple linear or your other favorite regression instead) is given in the "Calibration Methods" section of the attached paper. It is by no means the only way to skin the cat, just an example that's on hand in fresh in my mind since it came out recently.
Hope that helps!
Tony
Article Assessing the Impact of Retreat Mechanisms in a Simple Antar...
It is absolultely essential that level 1 heteroscedasticity is modeled otherwise you are likely to get biased estimates of the variances of the higher-level random effects in a multilevel model.
Here is an introductory paper
https://www.researchgate.net/publication/23538539_Modelling_complexity_Analysing_between-individual_and_between-place_variation_-_A_multilevel_tutorial
and for a detailed how to do it with MLwiN software see
https://www.researchgate.net/publication/260771330_Developing_multilevel_models_for_analysing_contextuality_heterogeneity_and_change_using_MLwiN_Volume_1_updated_June_2015_Volume_2_is_also_on_RGate
It shows you how to model variance as a function of observed variables at any level,
Article Modelling Complexity: Analysing Between-Individual and Betwe...
Book Developing multilevel models for analysing contextuality, he...
The first paragraph of the introductory paper that Kelvyn cited is great. I would add to it by suggesting that a part of the problem is that experimental designs are typically focused on predicting an average outcome. The four replicates I have for this activity does a spectacularly bad job of predicting the variance in the data.
Heteroscedasticity is also not a topic found in most introductory statistics courses except as a "problem" that needs to be corrected.
Sample size calculators focus on a sample size suitable for predicting an average outcome. Accuracy in estimating the variance is not discussed.
1) What am I looking at in Figure 1 in the tutorial Kelvyn? Shouldn't those distributions be vertical, not perpendicular to the line? The notation is different from what I'm used to seeing. Maybe I missed something.
2) Tutorial looks like what I was asking is in use, but I only see variance proportional to x, if I'm reading this correctly. That's often a good one though. [Oops. See next note, hours later, below please.]
...
Timothy - One comment to add to your last part: Yes, but without an idea of the population standard deviation, sample size requirements cannot be estimated, unless the 'calculator' is only for the worst case for proportions, simple random sampling. - Thanks.
...
An "Antarctic ice sheet model" sounds like a very interesting and useful application, Tony. Thanks.
Oops. Regarding comment 2 above, on the tutorial, actually an estimated residual term of format (x)(e) means variance is proportional to the square of x, not x, which is rather extreme heteroscedasticity. I did not expect that.
James
on 1 - I was trying to give a 3-d like view that around the line is a distribution of potential values - agree not the best representation!
on 2 in these models you can analyse the variance of Y as a constant, linear or quadratic function of X and the parameters are estimated so it is quite a flexible specification and a likelihood ratio test allows you to evaluate the degree of support for different specifications
It is also possible to model the Variance of Y as a function of Yhat. This is standard in Generalized multilevel models such as logit, probit Poisson, NBD . It is also possible to estimate a constant coefficient of variation model . On the latter see
http://www.stat.nus.edu.sg/~stachenz/ST5206Notes4.pdf
Ah. I was thinking of heteroscedasticity modeled as "...the Variance of Y as a function of Yhat." For example, Särndal, CE, Swensson, B. and Wretman, J. (1992), Model Assisted Survey Sampling, Springer-Verlang.
The issue with the variance of Y as a function of Yhat is I see that as a technical fix as it gives little insight into what is going on. Think of Xs as a dummies for two drugs- I would really be interested if one drug is much less variable in lowering blood pressure than the other. Modeling heteroscedasticity in this case as a function of the X's would be very valuable.
By technical fix I means that it is aimed at getting less biased standard errors.
Well, predicted y is a function of x's, and you can use other such functions (as on page 232 in Särndal, CE, Swensson, B. and Wretman, J. (1992), such as using just one of the regressors), but yes I'm more interested in prediction. You may try to analyze it further, though regressors cannot always be separated so well, but I would hardly call using predicted y as a size measure to be a "technical fix." However, I can see why you would want to try to do what you want to do, in your context, for your objective.
Kelvyn Jones noted that "It is also possible to model the Variance of Y as a function of Yhat." This relates to what Ray Chambers calls the "gamma population model," see page 49, Chapter 5, "Populations with Regression Structure," in An Introduction to Model-Based Survey Sampling with Applications, 2012,
Ray Chambers and Robert Clark, Oxford Statistical Science Series. To do this you would consider the size variable z to be predicted y, which would be best, according to my paper referenced next below. Considering "predicted y as a size measure" is 'essential' here:
https://www.researchgate.net/publication/320853387_Essential_Heteroscedasticity. This paper is a discussion of Brewer, KRW (2002), Combined survey sampling inference: Weighing Basu's elephants, Arnold: London and Oxford University Press, mid-page 111. There, in Ken Brewer's book, he explains the principle which one might use for this. Note that Särndal, CE, Swensson, B. and Wretman, J. (1992), Model Assisted Survey Sampling, Springer-Verlang also makes use of the gamma population model.
As explained by Ken Brewer, covered in https://www.researchgate.net/publication/320853387_Essential_Heteroscedasticity, this model makes sense. A larger value can be considered a cluster of smaller ones, and variance considered just as in Cochran(1977), Sampling Techniques, 3rd ed, Wiley, page 243. For more on this, see https://www.researchgate.net/project/OLS-Regression-Should-Not-Be-a-Default-for-WLS-Regression. This includes the following:
https://www.researchgate.net/publication/333642828_Estimating_the_Coefficient_of_Heteroscedasticity
And
https://www.researchgate.net/publication/333659087_Tool_for_estimating_coefficient_of_heteroscedasticityxlsx
I think the only reasons for failure of the "gamma population model" are model and data shortcomings, such as omitted variables, overfitting, and data quality issues.
Hello Kelvyn Jones do you perhaps know how to model level 1 heterogeneity using SAS proc mixed? Many thanks in advance!
Yu-Chin Her sorry I am not a SAS user, mlwin allows a level 1 variance function in which measured variables can be included.
- Badges
- Science topic
- Similar topics
- Statistics
- Survey