This effect is sometimes known as suppression. It is (in a sense) an issue collinearity in that the outcome of the regression depends on the interrelation between predictors and outcome.

I'm afraid you also misunderstand what multicollinearity is. If you have correlations between predictors you have some multicollinearity - whether the correlations are weak or non-significant is irrelevant. You have no multicollinearity if and only if the correlations between predictors are exactly zero.

A thought experiment: I have 10 predictors and each has a weak correlation with every other predictor of say 0.15. There are 45 correlations and the worst case scenario is that between them they account for no unique variance in Y. (r = .15 implies r^2 = .025 and 45 x .025 is greater than 1). So tiny amounts of shared variance between predictors can add up to high levels of collinearity.

In you case a VIF approach 2 indicates appreciable levels of collinearity because it implies that up to half the variance accounted for by X in Y is shared with other predictors,

So at one level there is nothing to explain. In fact it is the reason researchers use multiple regression at all: it gives different results to simple correlation. The real question is whether the regression model is appropriate for your research question.

e.g., I test people on speeded history test and find that males do better than females. I then include a measure of writing speed (e.g., perhaps because males write faster) and now females do better. This suggests males worse at history, but the original test confounds this effect with writing speed.

This effect is sometimes known as suppression. It is (in a sense) an issue collinearity in that the outcome of the regression depends on the interrelation between predictors and outcome.

I'm afraid you also misunderstand what multicollinearity is. If you have correlations between predictors you have some multicollinearity - whether the correlations are weak or non-significant is irrelevant. You have no multicollinearity if and only if the correlations between predictors are exactly zero.

A thought experiment: I have 10 predictors and each has a weak correlation with every other predictor of say 0.15. There are 45 correlations and the worst case scenario is that between them they account for no unique variance in Y. (r = .15 implies r^2 = .025 and 45 x .025 is greater than 1). So tiny amounts of shared variance between predictors can add up to high levels of collinearity.

In you case a VIF approach 2 indicates appreciable levels of collinearity because it implies that up to half the variance accounted for by X in Y is shared with other predictors,

So at one level there is nothing to explain. In fact it is the reason researchers use multiple regression at all: it gives different results to simple correlation. The real question is whether the regression model is appropriate for your research question.

e.g., I test people on speeded history test and find that males do better than females. I then include a measure of writing speed (e.g., perhaps because males write faster) and now females do better. This suggests males worse at history, but the original test confounds this effect with writing speed.

Mahyar -

Perhaps it is a matter of degree: say, a combination of other independent variables relates strongly enough to this variable in question, or perhaps that one variable alone would yield large estimates of the variance of the prediction error (low predictive 'power'), alone, so is not very useful.   

You could look at some test data and compare prediction performance with and without this independent variable in question.  It may not really even matter very much.  In that case, you might leave it out.  Omitting an essential independent variable can cause substantial bias, but the change in sign might indicate that would not be a problem here. 

(Nonlinearity might also be involved when getting a clearer picture of data relationships, by the way.)

Anyway, those are my speculations:    Looking at test data, and estimating the square root of the variance of the prediction error, may be helpful.  (Note that these estimates may include heteroscedasticity.)    

Cheers - Jim 

PS -  By comparing test data withheld y-values to those predicted with and without that independent variable in question, you can see performance.  Perhaps, graphically, you could plot known y on the x-axis, and the two predicted y values on the y-axis.  That might be informative.  It's a suggestion, anyway. 

Dear James, Thom,

I really appreciate your inputs. Really helpful.

Following James' suggestion, I compared the predictive power of the model with and without that variable, and only a slight improvement (negligible) was observed confirming the hunch that the variable might not be that important to the overall model(even though it was identified as statistically significant). However, I was wondering how to interpret the relationship between that variable (which changed its sign) with my dependent variable? If the observed sign in the regression model was just under the influence of the suppressor,  then, does it imply that I should mind its correlation coefficient's sign as the indicator of its true relationship? Or, as pointed out by Thom, its sign in the regression model might indicate its true relationship with the dependent when considering other variables, too. 

PS:

1) In my study, I need to compare the relative importance of multiple independent variables in determining my dependent (temperature) variable as well as the direction of their relationship. 

2) Partial correlation yields consistent results (in terms of sign) with regression mode, if it helps in any way. 

Mahyar -

Hi. -  You noted "...I need to compare the relative importance of multiple independent variables...."  That is, of its nature, a very nebulous concept.  Variables interact, and there is a question as to how nonlinearity should be considered.  The impact of each independent variable is not just hard to separate from the others, but it is generally going to be impossible.  For a sign to change as it did, however, I think you might often be able to say that the impact of that variable is negligible.  If you look at a regression (or regressions, also considering nonlinearity) using only that one independent variable, you will see the relationship that one independent variable has on the dependent variable, absent any other factors, but the variance of the prediction error for each predicted y-value is probably relatively huge. 

Anyway, those are my thoughts on this, if I haven't missed anything important. 

Cheers - Jim

Mahyar -

One more thing:  You noted "...that the variable might not be that important to the overall model(even though it was identified as statistically significant)."  If by "statistically significant," you are referencing a p-value, and an arbitrary threshold such as 0.05, without considering sample size, then this kind of 'cutoff' may not be very meaningful. 

The use of the word "significance" in statistics is often quite a misnomer. 

The following might help:

 https://www.researchgate.net/publication/262971440_Practical_Interpretation_of_Hypothesis_Tests_-_letter_to_the_editor_-_TAS

Best wishes - Jim

Article Practical Interpretation of Hypothesis Tests - letter to the...

As Thom pointed out, it's totally normal for the sign to change. There are many examples of ways this can happen, particularly if some variables are confounded. See e.g. https://www.researchgate.net/post/SPSS_Is_that_possible_the_obtained_Beta_value_in_Multiple_Regression_to_be_different_from_R_value_in_the_Correlation_Coefficients

1 Check the Covariance of response and that independent variable. 

2 Check the Covariance of response and that added variable.

3 You may know the reason. If covariance from step 2 is close to 0, there is collinearity; if covariance from step 1 is close to 1, then it happens due to it's effect.

Check interactions

Without any doubt it is the problem of multi-colinearity.  So first make sure all predictors are independent. 

Mahyar, 

In your case, pairwise correlations are not high but the additional contribution of your special regressor (x say) is small in the presence of the others, whereas it explains substantially when it is alone. This indicates some degree of collinearity involving at least three regressors (incl. x), and that you have a collinearity problem has been suggested also by all other replies. Your VIF for that regressor is "less than 2", which is not as high as I would have expected from the other observations above. However, you ask how your regression on x alone should be interpreted, and to answer that question it is easier to think of the more extreme case of exact collinearity, meaning that some particular linear combination of the regressors is fixed. Your situation is a bit on the way towards that extreme case, so it has relevance for how you should look at the problem. 

The simple regression on x tells how the response is expected to change with x when all others are kept fixed. But under a collinearity constraint involving x the others cannot be kept fixed, they must also change in order to satisfy the constraint. So the simple regression on x is an illusion, and misleading. And a question such as "which regressor is most responsible for the regression" is in principle impossible to answer from such data. 

There is an alternative possibility, that the collinearity is not a necessary constraint among the regressors but "only" due to the design behind the data, and that future data need not follow the collinearity restriction. But then the problem is that we have no information from data how this future deviation from the collinearity will influence the response, since we have not seen any such deviation. Therefore we cannot rely on the simple regression on x in this case, neither. 

I hope this helped to understand the problems you have experienced, even though they are not as serious as the extreme case above. 

Any relation between two variables is said to be spurious.  That is why, whenever we bring other independent variables explicitly into the regression model and we compute the partial regression coefficient (the effect of  X on   Y   after we remove the EFFECTS of other variables    we find that the algebraic sign of the simple relation may change.

In multiple regression we study the effect of X on  Y  KEEPING THE EFFECT   of other variables constant.