I did a multivariate regression analysis by SPSS. Some independent variables' coefficients were negative.But indeed, all coefficients should be nonnegative according to the practical meaning. So should i kick out all the independent variables with negative coefficients and redo the multivariate regression analysis untill all coefficients of remaining coefficients are positive?
Hello Yuqian,
The apparent "paradox" is due to redundancy/overlap/collinearity among the independent variables (IVs). If all IVs are uncorrelated with one another, then the magnitude and sign of regression coefficients will track those of the zero-order correlations (of each IV with the DV).
However, as you see with your data, it's quite possible that sign and/or magnitude of regression coefficient will not track those of the zero-order correlations. Why? Because the purpose of multiple linear regression is to identify the linear combination of IVs that yields the smallest sum of squared errors for that data set. Doing so, in the presence of collinearity, may well result in estimated regression coefficients that are not consonant with the zero-order correlations.
How to eliminate this? (1) Combine variables with strong correlations into factor/component scores; (2) Drop a variable which shows too-high redundancy (usual indicators are VIF or its inverse, tolerance) (3) Look at alternate techniques, like ridge regression.
How to live with, but not eliminate this? Report both the regression coefficient and the zero-order correlations. Doing so will help you avoid misrepresenting an IV as possibly being "Unimportant" based on its regression coefficient, if the zero order correlation with the DV is strong.
Good luck with your work.
@DavidMorse is completely correct but there's another possibility and that's the theory you are using may be incorrect. If this condition persists through Prof. Morse's advice be sure to check this out. This would then be a good of example of how many discoveries are made. Remember Richard Feynman a noble prize winner once said of theory if it doesn't agree with experiment then it's WRONG. Check your calculations etc but after that NEVER discount this last possiblity. In chemistry we say research is not like a lab manual. Good luck and may the force be with you. David Booth PS I have added an attachment that you might find useful in doing that which Prof Morse has suggested. Pay particular attention to the reference to Dennis Boos.
If you have access to to Mosteller & Tukey (1977) via your library, take a look at Chapter 13, Woes of Regression Coefficients. HTH.
https://www.pearson.com/us/higher-education/program/Mosteller-Data-Analysis-and-Regression-A-Second-Course-in-Statistics/PGM154255.html
- Badges
- Science topic
- Similar topics
- Mathematical Sciences
- Statistics
- Statistical Software
- SPSS