ORDINARY LEAST SQUARE

The ordinary least square is a method used to determine a regression line (straight line) that describes the relationship between X (independent variable) and Y (dependent variable). To obtain the linear equation, the following statements are used:

I = n(sum XY – (Sum X)(Sum Y)

II = n(Sum X^2) – (Sum X)^2

III = n(Sum Y^2) – (Sum Y)^2

b = I / II

a = Y^ - bX^

… where a = intercept; b = slope; Y^ = mean of Y and X^ = mean of X. The estimated equation under the ordinary least squares is given by:

(1) Y = a + bX

… where Y = simple regression line; a = intercept; and b = slope.

The estimate under equation (1) is generally seen as a biased estimate because each observed X may be weight differently. However, equation (1) does not take this weight into account. A corrective measure is to use the weighted variable.

WEIGHTED LEAST SQUARE

In order to use weighted least square, three pieces of information are needed: (i) observations of independent data set: {x = x1, x2, …, xn}; (ii) observations of dependent variable data set: (y = y1, y2, …, yn}; and weight assigned to each observations: {w: w1, w2, …, wn}. Note that the sum of weight must be 1.0. The following statements are used as guideline:

I = Sum w

II = Sum w(x)

III = Sum w(x^2)

IV = Sum w(y)

V = Sum w(x)(y)

D = I(II) – II^2

b = [III(IV) – IV] / D

m = [I(V) – II(IV)] / D

The estimated equation under the weighted least squares is given by:

(2) Y^ = b + mX

… where Y^ = estimated weighted least squares; b = intercept, and m = slope.

ACCURACY OF ESTIMATE

The accuracy of the estimated may be determined by the difference between the ordinary least square and weight least square, thus:

Y – Y^ > 0 means that OLS overestimated

Y – Y^ < 0 means that OLS underestimated

Y – Y^ = 0 means that OLS and WLS are equal (unlikely).

It is said that the weighted least squares estimate is better than OLS because each element of X is weighted.

CAUTION IN USING WEIGHTED LEAST SQUARE

The weight must be chosen by the researcher. All weight proportion must add to 1.00, i.e w1 + w2 + … wn = 1.00. In selecting the weight a rational standard must be used as to why a certain weight percentage is given to each element: x, less the weight assignment becomes arbitrary and the resulting Y^ may be different from what it should be. A second caution one needs to exercise is to be aware of where the weight applies: X or Y. In the example illustrated above, the weight is applied to both X and Y. It is also possible to apply the weight only to X or only to Y.

SOFTWARE

Follow the definition above using Excel spreadsheet can easily allow one to calculate WLS and graph the result. There is no need to use specialized software.

See Attached EXCEL for Weighted Least Squares.