Dear Nawaf, 

Check these papers out if they help you: 

Article An Improved Correction for Range Restricted Correlations Und...

Article Identification of asymmetric conditional heteroscedasticity ...

Article Robustness of Some Portmanteau Correlation Tests in Financia...

Article Variance-based interaction index measuring heteroscedasticity

Nawaf -

I do not know what you mean by "shoulder scores," and whether you have continuous data or other details of your application, but I did a great deal of work with simple regression with substantial heteroscedasticity, and I never felt heteroscedasticity had to be "corrected," but just used as part of the error structure:  

 https://www.researchgate.net/publication/262972023_HETEROSCEDASTICITY_AND_HOMOSCEDASTICITY

 https://www.researchgate.net/publication/263036348_Properties_of_Weighted_Least_Squares_Regression_for_Cutoff_Sampling_in_Establishment_Surveys

 https://www.researchgate.net/publication/263809034_Alternative_to_the_Iterated_Reweighted_Least_Squares_Method_-_Apparent_Heteroscedasticity_and_Linear_Regression_Model_Sampling

 https://www.researchgate.net/publication/261596397_Ken_Brewer_and_the_coefficient_of_heteroscedasticity_as_used_in_sample_survey_inference

Many want to "correct" so that they can use hypothesis tests, but there is often little to be gained by that.

 https://www.researchgate.net/publication/262971440_Practical_Interpretation_of_Hypothesis_Tests_-_letter_to_the_editor_-_TAS

Here is a press release on p-values from the American Statistical Association:

http://www.amstat.org/newsroom/pressreleases/P-ValueStatement.pdf

Your application may have unique needs, but you might consider weighted least squares (WLS) regression as above, if appropriate.

Cheers - Jim

Article HETEROSCEDASTICITY AND HOMOSCEDASTICITY

Article Properties of Weighted Least Squares Regression for Cutoff S...

Conference Paper Alternative to the Iterated Reweighted Least Squares Method ...

Article Ken Brewer and the coefficient of heteroscedasticity as used...

Article Practical Interpretation of Hypothesis Tests - letter to the...

Dear Dr. Béatrice,

Thank you for useful materials you have uploaded.

James,

Thank you for your answer and useful materials.

The shoulder scores are both continuous data ( The possible score for first score is 100 points and the second shoulder score is 48 points ).

I understand now heteroscedasticity could not be corrected but I would like to guide me whether I can find appropriate approach to deal with it or not. Do you think WLS will not affect on validity of the derived conversion formula ( regression equation ) ?. e.g. If clinician wants to convert from first shoulder score to the other shoulder score, he/she might be found different scores ( points might be out of the score ).

Yes, Nawaf, if I understand you correctly, and I think i do, WLS regression should work as well as if you did a transformation, but without the interpretation problems.  In either case, however, if you are near the limits of the range of your score, that would impact how a confidence interval could be done.

Anyway, I think you want to look at this:

 https://www.researchgate.net/publication/263036348_Properties_of_Weighted_Least_Squares_Regression_for_Cutoff_Sampling_in_Establishment_Surveys

For a single regressor and zero intercept,  see the bottom half of page 2 for b, and the bottom half of page 7 for variance.  If you have an intercept, look through section 7 for the coefficients and variance estimates.

Best wishes.  

PS -  Please note that for the estimates of variance of the prediction errors, I did that for both individual cases, and for estimates of totals from finite populations.  You want the individual ones. You can calculate them, as for the classical ratio estimator in the spreadsheet in

 https://www.researchgate.net/publication/263265199_CRE_Prediction_%27Bounds%27_and_Graphs_Example_for_Section_4_of_Properties_of_WLS_article

But I know that SAS PROC REG will give you the estimated square root of the individual variance of the prediction error as STDI.  (I suppose that stands for STanDard error for an Individual case. ???)  I imagine that R and other software will do something similar.  

A robust regression weight to enter might be w=1/x.   

Article Properties of Weighted Least Squares Regression for Cutoff S...

Data CRE Prediction 'Bounds' and Graphs Example for Section 4 of ...

An unequal variance of the residuals leads to inefficient slope standard error since it under-estimate or over-estimate, hence the t-values are biased. Often, it makes the slope to be statistically significant when it shouldn't  be. The WLS, iterative WLS, White standard errors and Newey-West estimators are some of the ways to minimize unequal variances. 

OLS is not biased, but it is inefficient (a reference to higher variance of b than WLS).  But this means there can also be a great impact on the estimate of these regression coefficients, even though unbiased over a large number of applications - which could be argued not to help you in a specific data application.  

Further, an impact of needing weighted least squares (WLS) regression is that you will not find the relatively much larger variances of the prediction errors for larger cases, compared to smaller ones, if you do not use WLS. 

I think that heteroscedasticity is often overlooked, so it is good to be aware of it.  Essential heteroscedasticity  (see below) is 'a feature, not a bug.'  OLS is a special case of WLS, and OLS is probably not often strictly appropriate.  For cross-sectional data, it is particularly inappropriate in general. 

The following documents may be helpful:

Essential Heteroscedasticity:

 https://www.researchgate.net/publication/320853387_Essential_Heteroscedasticity

Nonessential Heteroscedasticity:

 https://www.researchgate.net/publication/324706010_Nonessential_Heteroscedasticity

Very helpful views and links

Interpretation:

If we can treat these 'scores' as continuous variables, and have simple linear regression through the origin, we could graph something like that of the prediction interval endpoint-connected curves in https://www.researchgate.net/publication/263265199_CRE_Prediction_%27Bounds%27_and_Graphs_Example_for_Section_4_of_Properties_of_WLS_article, using a good coefficient of heteroscedasticity (gamma), where gamma = 0.5 was used there. 

A good gamma picked here would mean that plotting the random factors of the estimated residuals on the y-axis against x (or predicted y) on the x-axis would show that the nonrandom factor, here x^gamma (see bottom of page 1 in https://www.researchgate.net/publication/320853387_Essential_Heteroscedasticity: y=bx+(e0)(x^gamma)) does capture the appropriate degree of

heteroscedasticity ... though the prediction intervals would actually be substantially skewed near the origin, because negative values would not be appropriate. 

Note also that if this is actually a linear relationship, graphical residual analysis would also show that.

If b is the "conversion" factor here, assuming a zero intercept, then the WLS estimate of b is more accurate than OLS, and you can see the increased variance of y for a given x that you want to "convert," as you get to larger x values. 

If there is a distinct, non-zero intercept, then you may have missing variables (https://www.researchgate.net/publication/324706010_Nonessential_Heteroscedasticity).  That would indicate more and/or different information in one 'score'/measure than the other measure. 

Anyway, just some thoughts.  Cheers. 

I longer see heteroskedadcity a problem since there are numerous ways to handle this!

I don't think heteroscedasticity can be just dismissed as a problem that has solutions.  A reasoned approach is needed, yet often not applied.  As shown in https://www.researchgate.net/publication/324706010_Nonessential_Heteroscedasticity, some causes of heteroscedasticity somewhat artificially produce it, and can be addressed to a degree - though, as an analytical chemist wrote to me recently, measurement error in different ranges to different degrees can be the primary problem, which also can happen in survey statistics, or, as I imagine may occur in many applications, a missing variable may be unknown or data may not be available for it.  (If you could find variables to create perfect correlation with no measurement error, then the residuals would be zero, which is not happening.) 

For time series and other cases involving autocorrelation, this becomes more complex, and generalized least squares (GLS) regression is needed.  I confine my remarks to cross-sectional population surveys.  (My experience is largely in inference from sampling of finite populations.) 

In addition, there is the heteroscedasticity which is an integral part of the error structure.  Even when there are no special problems leading to heteroscedasticity which one might do any more to ameliorate, one still has to consider the varying sizes of members of the population.  Thus, in the case of "essential heteroscedasticity," the variance structure inherent in considering each member of a population to be a cluster of a different size is discussed.  (See 3rd ed of Sampling Techniques by William Cochran, 1977, Wiley, page 243, under "Single-Stage Cluster Sampling," and https://www.researchgate.net/publication/320853387_Essential_Heteroscedasticity.) 

Then we have to consider that one very popular way people have handled this, at least in econometrics, has been to divide through a regression equation by the inverse square root of the regression weight, which is the nonrandom factor of the estimated residuals, so as to use the resulting hard-to-interpret equation to then apply a nebulous hypothesis test, often only looking at an isolated p-value, to then come to an unjustified and often misleading 'conclusion.'  Rather, one should use weighted least squares (WLS) regression, leave the heteroscedasticity in the error structure where it naturally belongs, and then look at prediction intervals, or just the square root of the variance of the prediction error in various cases, not hypothesis tests. 

The treatment of heteroscedasticity does require proper attention, is apparently routinely mishandled, and should not be dismissed as unimportant to discuss.  It is also far more ubiquitous, at least in population cross-sectional surveys, than seems to be generally recognized.  OLS is a special case of WLS, and at least for population cross-sectional surveys, OLS should be considered the exception, not the rule. 

Note also that we may be looking at more than prediction.  See the example in the appendix to Galit Shmueli's paper at the following location: https://www.researchgate.net/publication/48178170_To_Explain_or_to_Predict

We can use WLS to look at confidence intervals of regression coefficients, but note that interactions such as collinearity very much muddy the water. 

Jim, you have lots of time on your hand these days so enjoy it, but your comments above, including the ones on cross-sectional data (your area of experience) aren't convincing. You started out feeling that heter.( unequal variance) should not be corrected but should be used as part of the error structure. Then, after ziging and zaging, you are agreed that WLS or ITWLS is good. Then, the young lady kept telling you that her data are interval in nature. Unfortunately, you did not address her issues, instead you recommended several papers she needed to read.

From there, you went on to dream about intercept and origin issues that she never asked for. Who focuses on a "garbage in garbage out" term or an estimated equation thru the origin. From theoretical and empirical analyses, we know that some important statistics rely on a model having an intercept term.As I stated earlier, please, I no longer see heteroskedadcity as a problem and there are numerous ways to handle it. Like any scientific field. there is usually no cure, but there ways to ways to minimizing their effects: WLS and numerous others I mentioned early do this. When have you interpreted a constant term in any of your recent referred publications? I will guess zero! Bear in mind, that equation can be logged and FGLS estimator will be applicable since beta is unbiased. If prediction is her goal (WHICH IS HIGHLY UNLIKELY), collinearity is not an issue. It is well-known known that explanation is most important in regression analysis. One needs to understand that their model met all relevant OLS/WLS assumptions before even thinking about prediction (oh well sleepy!!).

Nawaf is predicting one measure from another using a regression coefficient (b) as a "conversion" factor.  WLS should be used with the heteroscedasticity in the error structure.  I hope I have now stated that more clearly.  

A spreadsheet tool for estimating or considering a default value for the coefficient of heteroscedasticity, developed for linear regression, is found here (with references):

https://www.researchgate.net/publication/333659087_Tool_for_estimating_coefficient_of_heteroscedasticityxlsx

In order to move from OLS to WLS regression, one may pick a coefficient of heteroscedasticity from the above, and use that to find a regression weight expression.  The regression weight would be "w" entered into SAS PROC REG, for example. 

Note that when the coefficient of heteroscedasticity is larger than 0, the estimated variance of the prediction error will be larger for larger predicted y values.

When the coefficient of heteroscedasticity is zero, regression weights are constant.