I encountered a problem while performing multiple linear regression and ols single factor regression. "There are two factors with low coefficients when performing single factor regression, but when performing multiple factor regression, the coefficients of these two factors are also high.". After removing one factor and performing multifactor regression, the coefficient of the other factor also decreased significantly. But. These two factors have passed the collinearity test and are not collinear. Therefore, why is there such a result? Are these two factors good at fitting the equation？
How large is your sample size, and how large is the correlation between the two predictors?
You seem to be describing suppression.
- https://statisticalhorizons.com/when-do-suppressor-effects-occur/
Depending on the nature of your causal model, other names for the phenomenon you are seeing could be negative confounding or inconsistent mediation. See this article for more info:
MacKinnon, D. P., Krull, J. L., & Lockwood, C. M. (2000). Equivalence of the mediation, confounding and suppression effect. Prevention science, 1, 173-181.
Article MacKinnon DP, Krull JL, Lockwood CM. Equivalence of the medi...
Thanks your answer. The sample size is about 50 000. I conducted a Pearson correlation test according to your reminder. The correlation between the two was 0.18, with a low correlation, while the value of linear regression beta was maximum. I wonder if such a small correlation coefficient indicates that the two variables are not linearly correlated and not suitable for linear regression, although the significance of the two variables is 0, < 0.001. maybe the relationship between the two variables is random. Is there any theoretical basis to prove that two factors with low correlation are not suitable for linear regression?@https://www.researchgate.net/profile/Christian-Geiser
Lijian Sun Thanks for providing additional information. To understand what is going on, it would be necessary to see the entire covariance and/or correlation matrix for all three variables as well as the regression results (coefficients). Also, as David Eugene Booth mentioned, looking at the bivariate scatter plots for all three variables may help.
Thank you for your reply. I apologize for the late reply, it is because I rechecked the calculations and updated the data to ensure rigor. Figure 1 and 2 are scatter plots of the two independent variables with Y. Figure 3 is a scatter plot of the two independent variables, and Figure 4 is a scatter plot of the product of the two independent variables with Y. The correlation coefficients of X1 and X2 with Y are -0.25 and 0.24, respectively. the correlation coefficient of X1 with X2 is 0.73. their regression equations h and R2 are as follows:
1. Y= -0.039 X1+0.21; R2 = 0.062;
2. Y = -0.038 X2+0.214; R2 = 0.059;
3. Y= -0.146 X1 + 0.146 X2 + 0.214; R2=0.455.
The correlation between the two independent variables and Y is not very low, but their scatter plots do not have a significant linear relationship, are they not suitable for regression analysis?
The correlation between the two independent variables and Y is not very low, but their scatter plots do not have a significant linear relationship, does that mean they are not suitable for regression analysis? I look forward to your reply.@https://www.researchgate.net/profile/David-Booth-7
- Badges
- Science method