Hi,
I would like to calculate after running linear regression relative weights for each predictor variables. As the most applied technique in this case is using standardized Beta coefficients - summing them up and calculating contribution of each variable. However, I found paper Bring 1994 (How to Standardize Regression Coefficients) which questions the reliability of standardized beta and suggest to square t statistics, sum them up and find the relative contribution of each variable.
Do you have any suggestions/ideas regarding using squared t statistics?
Best regards,
Davit
For the benefit of others, I think Davit is referring to this article:
How to Standardize Regression Coefficients
Johan Bring
The American Statistician
Vol. 48, No. 3 (Aug., 1994), pp. 209-213
I have not (yet) read that article, so cannot comment on what Bring says. But regarding standardized coefficients, I like what John Fox says about them in his book Applied Regression Analysis and Generalized Linear Models (2008, Sage). See the Google Sites link below for my notes on what Fox said. HTH.
https://www.jstor.org/stable/2684719?seq=1#page_scan_tab_contents
https://sites.google.com/a/lakeheadu.ca/bweaver/Home/statistics/files/Fox%20on%20standardized%20regression%20coefficients.txt
Dear Bruce,
Thank you for your comment.
Bring has empirically showed inconsistency in using beta standardized coefficients for calculating relative importance of each independent variable. The problem he highlights is related to the formula how standardized beta is calculated.
If standardized coefficients were correct indicators of relative importance, then eliminating the variable with the largest standardized coefficient from regression should cause the largest reduction in R2 intuitively.
Standardized Beta1=B1*sd(X1)/sd(Y)
This is the formula for calculating standardized beta coefficients. Here, B1 is the unstandardized coefficient, which is multiplied by the standard deviation of X1 divided by the standard deviation of Y. The inconsistency stems from the following: B1 is the partial effect of X1, but sd(X1) is the overall standard deviation of X1, not partial. This means that we multiply two things, which are unrelated.
If we want to get correct standardized beta coefficients, we need to multiply unstandardized coefficients with partial standard deviations (rather than full standard deviation) of respective independent variables and divide by the standard deviation of the dependent variable. Partial standard deviation=standard deviation/(sqrt(VIF)) * sqrt(n-(1/n)-k).
It has been proved that in terms of new standardized coefficients, B1/B2=t1/t2.
In other words, instead of comparing new standardized coefficients, we can just compare t values. The squared t value measures the increment in R2 that is obtained by including the variable as the last variable in the regression equation.
Comparing t values is equivalent to considering the reductions in R2, obtained by eliminating each of the variables.
- Badges
- Science topic
- Similar topics
- Philosophy
- Esthetics
- Interpretation