In joint hypothesis testing, we normally estimate the unrestricted model and note the RSS value. Similarly, we estimate the Restricted model and note the RSS. Why does the RSS of the restricted model increase if the null hypothesis is false?
Thank you.
Jochen,
this is simply the best explanation and answer to the question I could possibly imagine.
Thank you a lot as I will use it myself now, to explain it.
The restrictions in the model reduce the dimensions of the subspace used to generate the predictions for your dependent variable. Thus, with the restricted model you have less information to generate the predictions for your dependent variable. The exception is the one mentioned by Jochen.
Dear Dennis,
Addition of restrictions to the model (not necessarily it is a linear regression) at least, the residual sum of squares does not decrease. This is general property of a solution of any optimization task. If the residual sum of squares is increase, some restrictions reduce in exact equalities.
If there are restrictions, parameters estimates are not normal even when normal noise in a regression.
I can recommend my book on this problem P.S. Knopov, A.S. Korkhin. Regression Analysis Under a Priori Parameter Restrictions. Springer, NY, 2011.
The solution of the ordinary least squares (OLS) regression is the set of coefficient values for which the residual sum of squares (RSS) is minimal. Therefor, there is by definition no other combination of coefficient values that would give you a smaller RSS.
If the residual sum of squares is increase, some restrictions reduce in exact equality.
If you restrict the model by holding one of the coefficients at a given constant you will get a different combination of the coefficient value.
- Badges
- Science topic