What are the various statistical software systems available which will allow you to enter regression weight, w, in weighted least squares (WLS) regression, in addition to SAS?
.
Thank you to Guillermo Enrique Ramos for confirming, in another thread/question, that SAS can be used for quadratic linear regression, in addition to straight line linear and multiple regression, to implement weighted least squares in the format y = y* + (e0)(w^(-0.5)), where w is the regression weight as shown in section 2 of "Estimating the Coefficient of Heteroscedasticity," https://www.researchgate.net/publication/333642828_Estimating_the_Coefficient_of_Heteroscedasticity. That is the format for which Ken Brewer made a convincing argument when describing the range of heteroscedasticity one should expect, as discussed in "Essential Heteroscedasticity," https://www.researchgate.net/publication/320853387_Essential_Heteroscedasticity. In section 3 of "Estimating the Coefficient of Heteroscedasticity," one can see that another way of handling heteroscedasticity is sometimes used, but it is not consistent with the Brewer explanation of the root cause of naturally occurring heteroscedasticity. Perhaps that alternative technique, or something related to it, might be used in some other software systems.
Here, the two attached images show the format for the regression I think best to use, and the regression weight, w, both presented in terms of the coefficient of heteroscedasticity, gamma. y* is the WLS prediction. In most applications, the best practical/obtainable function values for the size measure, z, are the OLS predicted y values. That is what is suggested for input to the spreadsheet in https://www.researchgate.net/publication/333659087_Tool_for_estimating_coefficient_of_heteroscedasticityxlsx.
(Note that the method here, using the attached images, is in completely closed form in the case where we have one regressor and a zero intercept, so z = bx, and x can be used for a (relative) size measure. Otherwise, we first obtain OLS predictions, and then pick a reasonable coefficient of heteroscedasticity, and go back to find the improved WLS predictions. I think that the other method appears to be more ad hoc.)
So what other software, in addition to SAS, will allow the input of w for weighted least squares regression? Is it for linear and multiple linear regression? Note that such software would then minimize the sum of weighted squared estimated residuals with respect to each regression coefficient, simultaneously, to obtain estimates of regression coefficients. What about polynomial linear regression? Multiple regression with interaction terms? Will it handle those too?
The other question I wrote regarding polynomial regression is still open, but here I'd like to know about other software for any regression fitting the format in the images attached here.
.
Please see https://www.researchgate.net/project/OLS-Regression-Should-Not-Be-a-Default-for-WLS-Regression.
.
The question then is What other software, besides SAS, will accept regression weight input, w, for WLS regression?
Thank you.
R allows to fit weighten linear and generalized linear models:
https://stat.ethz.ch/R-manual/R-devel/library/stats/html/lm.html
https://stat.ethz.ch/R-manual/R-devel/library/stats/html/glm.html
Note the argument "weights" that can be passed to the function. The argument is explained in "lm" as:
weights: an optional vector of weights to be used in the fitting process. Should be NULL or a numeric vector. If non-NULL, weighted least squares is used with weights weights (that is, minimizing sum(w*e^2)); otherwise ordinary least squares is used. See also ‘Details’.
And in the Details, following information is given:
Non-NULL weights can be used to indicate that different observations have different variances (with the values in weights being inversely proportional to the variances); or equivalently, when the elements of weights are positive integers w_i, that each response y_i is the mean of w_i unit-weight observations (including the case that there are w_i observations equal to y_i and the data have been summarized). However, in the latter case, notice that within-group variation is not used. Therefore, the sigma estimate and residual degrees of freedom may be suboptimal; in the case of replication weights, even wrong. Hence, standard errors and analysis of variance tables should be treated with care.
"Non-NULL weights can be used to indicate that different observations have different variances (with the values in weights being inversely proportional to the variances."
That sounds like the alternative I mentioned in section 3 of "Estimating the Coefficient of Heteroscedasticity."
Basically, the difference is in how results are smoothed. If done as shown in the attached images, then the nature of heteroscedasticity is explained in "Essential Heteroscedasticity," noted in the question description. Each y-value is considered to be a cluster of minutely small y-values, where the variance within the cluster is shown to account for the heteroscedasticity. The predicted value for the cluster is the size measure I noted.
Sections 2 and 3 of "Estimating the Coefficient of Heteroscedasticity," as noted above, should clear this up for you.
So I'm asking about a regression weight, w, as in the attached images.
Perhaps what is attached here will be more familiar to Econometricians.
Note that if z = x, y* = bx, and gamma is 0.5, then you have the usual ratio estimator. (For more complex predictions, gamma may be estimated using a preliminary predicted y, say the OLS predictions, to then have the WLS predictions. Note that OLS is a special case of WLS which is often assumed for ease when not realistic.)
I just want to know, Where, besides SAS, can you enter a "formula" for regression weight, w, such as w = 1/x?
Thank you.
Abolfazl Ghoodjani -
I have not used SPSS in many, many years. So can you enter a function for w somewhere? Where would you do that? A little detail please. Thank you very much. - Jim
PS - I mean, for example, you could go to SAS PROC REG, and enter w, for linear and multiple regression, but not for polynomial regression, unless perhaps if you do something special, as Guillermo suggested. He said one can go to SAS PROC GLM for quadratic regression. What about SPSS?
You know that I want to know if a function, an expression for w, can be entered, not just an ad hoc vector of values, as explained above, right? This means use of the coefficient of heteroscedasticity.
I want to know which statistical/econometric software systems support this.
Attached is an illustration I put together to show the basic idea of the use of the coefficient of heteroscedasticity in these regression weights. Perhaps that will make this more clear.
Data Desk (www.datadesk.com) allows weights in any regression or general linear model.
Thanks Paul. Let me be sure that we are talking about the same thing, please: You mean one can enter an expression for regression weight, w, such as 1/x, or x^(-1.5), or for multiple regression, say, (3.77x1 + 2.42x2 - 0.86x3)^(-1.7), where 3.77x1 + 2.42x2 - 0.86x3 was a preliminary result, right?
(Note: In the three examples above, the coefficient of heteroscedasticity, gamma, is 0.5, 0.75, and 0.85, respectively.)
Also, what about polynomial regression?
Thanks.
Data Desk allows you to define a "variance" variable (the reciprocal of the weight) and introduce it into the analysis. That variable can be defined with a formula involving any other variables available. However, it will choke if the definition is circular. Of course, you can evaluate the variance expression into numbers and then use it.
Thank you Paul. It sounds like there is a chance to use regression weights as I've noted, and as used by Ken Brewer, and proportionally by Sarndal, Swensson and Wretman, and Royall, Anderson, etc. It has meaning for the estimated residuals, and should be expected based on the size of each prediction, as discussed in "Essential Heteroscedasticity," linked above. (Note references there to Brewer and to Cochran, especially the middle of page 111 in Brewer's book.) It appears that many people use WLS on a more ad hoc basis. Perhaps many even just think of WLS as a way to handle outliers, but I am referring to natural/"essential" heteroscedasticity that should be expected, though impacted by model specification and data issues.
Anyway, thank you for your notes.
Please see the current update to this project:
https://www.researchgate.net/project/OLS-Regression-Should-Not-Be-a-Default-for-WLS-Regression
The title of that update is "Regression weights and statistical software."
Jean-Pierre -
Thank you.
Question:
So, for example, in the case of the model-based approach to the classical ratio estimator, the coefficient of heteroscedasticity, gamma, in this case, is 0.5. Since regression weight, w, in this case, is
w = x^(-2 gamma), then
w = 1/x.
So, do you know, will R let w=1/x be entered, for example, just as it is in SAS PROC REG?
Does anyone know about that?
Thanks - Jim
- Badges
- Science topic
- Similar topics
- Mathematical Sciences
- Statistics
- Statistical Software