Better to go by the logic.  If there is interaction, it must be there in the model.  Don't bother much about collinearity. Some times you can leave it as such.

Detecting multicollinearty means at least two variables are highly correlated (>.6) or measuring same things. You can omit one of them, not both. Interaction is no matter, you can report it.    

Hi Zerihun

Collinearity is a concern. As you may have known, multicollinearity can affect your discussion and may negatively affect your conclusion. This is because the 'actual' effect of two or more variables can't be determined for certainty. Think when you are in a hall where so many people talk at the same time, you can hear the talking but you can't determine who says what.

Following Abdolghani's recommendation, you can either drop one of the variables or combine them as 1. General cut-off points for VIF:

ideally

Agree with post above - also might be worth looking at ridge regression. Available as a syntax in spss. 

Hi Zerihun,

I published an article at the end of 2013 in which I developed an approach that deals with the issue you raise.  This new approach, which I call Sequential Residual Centering (SRC), eliminates the portion of multicollinearity called "inessential ill-conditioning," which is the multicollinearity among the component terms of an interaction.  For instance, for the highest-order interaction xwz, inessential ill-conditioning would be the multicollinearity among the following related terms (x, w, z, xw, xz, wz, xwz).  [Of course,  these terms may also be characterized by "essential ill-conditioning," which refers to their multicollinearity with other specified predictors that are not also component terms of the interaction.  The portion of multicollinearity that contributes to "essential ill-conditioning" cannot be eliminated in SRC, however I also reported in my article an extended version of SRC I created (SRC-CS) that does also allow conditioning essential ill conditioning from control variables or secondary predictors]. 

When VIF values are inflated, my SRC approach obtains the same regression slope values but lowers both the standard errors and the VIF (and therefore the p values) for the component terms of an interaction, which improves the sensitivity and statistical conclusion validity of moderated regression (i.e., reduces Type II error).  I conducted simulations which validated the SRC under a variety of conditions.  SRC and SRC-CS are relatively easy to understand and conduct using any regression software.

The citation for my article is:  Francoeur, R. B. (2013). Could sequential residual centering resolve low sensitivity in moderated regression? Simulations and cancer symptom clusters. Open Journal of Statistics, 3, 24-44.  You can locate and download it under my profile on ResearchGate (https://www.researchgate.net/profile/Richard_Francoeur) or in the December 2013 issue of the Open Journal of Statistics (http://dx.doi.org/10.4236/ojs.2013.36A004).

Let me also add that an interaction EFFECT is based not only on the interaction TERM, but also on all of the related lower-order terms that make up the interaction.  All of the terms should be retained; there will be specification error if you were to drop any of these related lower-order terms.

I agree with other respondents that ridge regression may also be an option, which you may want to specify initially without the interaction term.  In some situations with interactions, however, it may be more difficult than SRC or SRC-CS to execute (e.g., see http://stats.stackexchange.com/questions/68463/should-i-include-interactions-in-ridge-regression). 

One recommended solution to get results from ridge regression in these circumstances is to use increasing penalization (an option in R but not other statistical programs) for linear main effects, nonlinear main effects, linear interaction effects, nonlinear interaction effects (again, see http://stats.stackexchange.com/questions/68463/should-i-include-interactions-in-ridge-regression), however I am unsure about whether these results would remain sufficiently valid, especially if you hope to probe the nature (magnifying and/or buffering effects) at different points along the interaction.  I say this because, as I mentioned in the second-last paragraph, an interaction EFFECT is based not only on the interaction TERM but also on all of the lower-order terms that make up the interaction -- since the interaction effect is held within multiple terms, it is unclear to me whether increasing penalization across the increasing orders of terms would provide sufficiently valid results for post-hoc probing of interaction effects along different points of the interaction.

Best,

Richard

P.S. I also have two methodology articles that are in press in the online journal, OncoTargets and Therapy.  Both articles apply SRC to detect cancer symptom interactions and should appear online in sequence (one article is Part I, the other is Part II) within a month or so.

Multicollinearity causes problem in regression. If two or more predictor variables in a multiple regression model are highly correlated, the coefficient estimates of the multiple regression may change erratically in response to small changes in the model or the data.  With and without one of them, the coefficients for some correlated factor may change sign, may change magnitude; the standard error will increase a lot. Say x1, x2 and x3 are correlated with a linear relationship.  Without x3, the model fit well, both x1 and x2 are significant, adding x3 in the model, the model fitting will not get improve, the standard error for the regression coefficient will be much higher, the regression coefficients for x1 and x2 will change a lot. See the following example, x1= doprod, x2= stock, x3 =consume.  The first model includes  x1 and x2 and the third one includes all 3. It can be seen the change on coefficients and standard error.

Parameter Estimates

Variable  DF  ParameterEstimate StandardError t Value Pr > |t|

Intercept 1  -8.44014              1.43518               -5.88    0.0004

STOCK 1   0.62248               0.12787                4.87     0.0012

DOPROD  1      0.14531              0.00703         20.67  |t|

Intercept     1     -8.8227E-16               0.03244          -0.00    1.0000

DOPROD   1     -0.33934                   0.46402           -0.73   0.4883

STOCK       1    0.21305                 0.03434             6.20    0.0004

CONSUME  1    1.30268               0.46416             2.81    0.0263

 However multicollinearity does not reduce the predictive power or reliability of the model as a whole. It only affects calculations regarding individual predictors. That is, a multiple regression model with correlated predictors can indicate how well the entire bundle of predictors predicts the outcome variable, but it may not give valid results about any individual predictor.

Find collinearty, you may use principal component analysis. I fuse PCA, for the above example, one of the eigenvalue is almost zero,  the related eigenvector score is C3=0.707X1+0.007X2-0.707X3, this means x1 is almost equal to x3, you may drop consum or doprod, or use new= consume + doprod in the model.

You have received a lot of good advice from the discussants above. I regularly teach  graduate statistics courses to students majoring in Mathematics & Statistics, in addition to teaching courses on Biostatistics and "Statistics for Psychology" to graduate students. I would first get a good grip on the multicollinearity before trying to model the interaction (which is straightforward and not a problem).

As for multicollinearity, the "cut-off" is not a set rule. It is a rule of thumb. In some data sets, models that appear to have excellent fit (R square > 0.90), in fact include variables that are infested with multicollineairtiy. Prediction could be way off. In other instances, the "best" models for fitting are also great for prediction.

If you want to keep all regressor variables in the modeling, you need to consider using Ridge Regression. A small perturbation (k) is added to the data so that the very small eigenvalues (due to multicollinearity) are bumped up, and a small amount of bias is induced. In return, the regression properties improve with respect to the prediction capabilities of the model and also in terms of the estimation of the regression coefficients which often are inflated in magnitude due to multicollinearity.

Good luck with your work

Thank you all for your valuable insights and for taking taking time to respond to my question.  I must say I'm very glad to have joined Research Gate.

If you want to keep all regressor variables in the modeling, you may use the 2-parameter estimator defined by Lipovetsky and Conklin (2005) (http://onlinelibrary.wiley.com/doi/10.1002/asmb.603/abstract) which has a better performance than Ridge estimator has. It maximizes the quality of regression fit as well.

What I find very convenient is using PCA, as recommended also by Yuanzhang Li. A PCA on the 22 variables yield also 22 PC’s, ranging from very important, PC1, to insignificant, PC22. I would retain and continue calculations only with the most relevant PC’s. In other words, use only those PC’s that explain say >= 95% of the total variation and skip the rest. Usually this leaves you with much less PC’s. A PC analysis on the correlation matrix effectively eliminates multicollinearity and results in PC’s that are orthogonal ie are linear independent of each other. Since each PC is a linear combination of the original variables they can be calculated back to the original variables after performing a MLR on the PC’s. Thus,

PC1 = a1 x A + b1 x B  ….. v1 x V              (x is the multiplication sign)

PC2 = a2 x A + b2 x B  ….. v2 x V

Etc

Run MLR on the PC’s.

This yields,

Response = coef1 x PC1 + coeff2 x PC2……………+.coeffx x PCx   (where x < 22)

Now calculate back to original variables A through V

This yields,

Response = coeffa x A  + coeffb x B……………+.coeffv x V

Now the effect or contribution of the each variable can be evaluated. If small, discard it from the equation. Based on a residual analysis you can try to incorporate interaction terms, such as A x B,or even quadratic terms. But only if this necessary and you have sufficient degrees of freedom.

I hope this helps.

This URL will be helpful for you.

http://www.google.iq/url?sa=t&rct=j&q=&esrc=s&source=web&cd=2&ved=0CCsQFjAB&url=http%3A%2F%2Fwww.uk.sagepub.com%2Fsalkind2study%2Farticles%2F15Article01.pdf&ei=jklJVLupJoGw7Aa6m4DgCA&usg=AFQjCNG5nGuGcoOlb00BPmdDqzeOSzaeEQ&sig2=69KnZFMzL3q0seiuW-RBkA&bvm=bv.77880786,d.ZGU

according to PAUL D. ALLISON, when "the high VIFs are caused by the inclusion of powers or products of other variables," "you can Safely Ignore Multicollinearity." please see http://statisticalhorizons.com/multicollinearity. However, can anybody provide a formal reference?