Dear Syed Muhammad Amir,

First I'd like to explain what the Multicollinearity mean. Multicollinearity occurs when your model includes multiple factors that are correlated to each other instead just to response variable. In other words, it results when you have factors that are a bit redundant. So hide  the influence that each factor has on response variable. Secondly, I would like to explain how you recognize whether there is multicollinnearity or not. You can easily identify the existence of Multicolinearity by displaying the correlation matrix of continuous independent factors so you can note the existence of multicolinearity with  variables who has high correlation among themselves. 

Indications/Signs of Multicolinearity:

1. A regression coefficient is not significant yet theoretically, that variable should be highly correlated with response.

2. When you add or delete a factor from your model , the regression coefficients change dramatically.

3. You may see a negative regression coefficient when your response should increase along with  that factor and vise versa.

4. Your independent variables have high pairwise correlations.

Therefore, In the multiple linear regression  analysis, we can easily check multicolinearity by clicking on diagnostic for multicollinearity (or, simply, collinearity) in SPSS of Regression Procedure. However, for logistic we don't have that option. But we can solve this problem by using multiple linear regression for the set of independent factors excluding the original response and letting one of the factors as response and check the multicolinearity. Then use the variables with less multicolinearity as the independent factors for the original response. Second option Use Principal Component Analysis / Factor analysis. Third use Scatter plot/pair matrix plot and also correlation matrix of the factors. While you are doing all these techniques  you must create dummy variable for categorical independent factors since SPSS do not accept categorical predictors for this procedures. 

One way to measure multicollinearity is the variance inflation factor (VIF), which assesses how much the variance of an estimated regression coefficient increases if your predictors are correlated.

A VIF between 5 and 10 indicates high correlation that may be problematic. And if the VIF goes above 10, you can assume that the regression coefficients are poorly estimated due to multicollinearity.

Finally, either you can exclude the one or more factor with high multicolinearity or you can group in to one meaning full class for those are highly correlated variables by using either Principal component analysis(PCA) or Factor analysis (FA) to reduce the correlated variables from the model.

I hope this will be helpful to answer you question and minimize your challenge

.

Dear Syed,

What you should do is turn your dependent variable into a scale variable then run a linear regression and can check the multicollinearity diagnostics. VIF values > 5 are a concern.

Syed, as Peter says, you can just estimate your model with the REGRESSION procedure and select the available multicollinearity diagnostics.  I'm not sure what he means by turning your DV into a scale variable.  The REGRESSION procedure is an older procedure that doesn't make use of meta-data about the variables, so you should be able to just plug in your dichotomous DV.  The reason you can do this is that the multicollinearity diagnostics are concerned with the explanatory variables only, and will remain the same even if you use a randomly generated variable as the dependent variable.  (Give it a try if you don't believe it--just make sure you use exactly the same cases in the analysis.)

HTH.

Dear Amir,

Many solutions are here:

http://blog.minitab.com/blog/understanding-statistics/handling-multicollinearity-in-regression-analysis

And excellent videos are here:

https://www.youtube.com/watch?v=I4z3yjoEADY

https://www.youtube.com/watch?v=oPXjQCtyoG0

If you can find any two variables with multi-collinearity, you can delete any of them from your multivariable logistic regression analysis.

Dear All,

Actually, My dependent variable is dichotomous i.e. BGT Adoption (Adopted / Rejected). And I already applied binary logistic regression. Now, how can i find multicollinearity among all independent variables? Is it the correlation in  binary logistic regression output results in SPSS?

Dear Syed Muhammad Amir,

First I'd like to explain what the Multicollinearity mean. Multicollinearity occurs when your model includes multiple factors that are correlated to each other instead just to response variable. In other words, it results when you have factors that are a bit redundant. So hide  the influence that each factor has on response variable. Secondly, I would like to explain how you recognize whether there is multicollinnearity or not. You can easily identify the existence of Multicolinearity by displaying the correlation matrix of continuous independent factors so you can note the existence of multicolinearity with  variables who has high correlation among themselves. 

Indications/Signs of Multicolinearity:

1. A regression coefficient is not significant yet theoretically, that variable should be highly correlated with response.

2. When you add or delete a factor from your model , the regression coefficients change dramatically.

3. You may see a negative regression coefficient when your response should increase along with  that factor and vise versa.

4. Your independent variables have high pairwise correlations.

Therefore, In the multiple linear regression  analysis, we can easily check multicolinearity by clicking on diagnostic for multicollinearity (or, simply, collinearity) in SPSS of Regression Procedure. However, for logistic we don't have that option. But we can solve this problem by using multiple linear regression for the set of independent factors excluding the original response and letting one of the factors as response and check the multicolinearity. Then use the variables with less multicolinearity as the independent factors for the original response. Second option Use Principal Component Analysis / Factor analysis. Third use Scatter plot/pair matrix plot and also correlation matrix of the factors. While you are doing all these techniques  you must create dummy variable for categorical independent factors since SPSS do not accept categorical predictors for this procedures. 

One way to measure multicollinearity is the variance inflation factor (VIF), which assesses how much the variance of an estimated regression coefficient increases if your predictors are correlated.

A VIF between 5 and 10 indicates high correlation that may be problematic. And if the VIF goes above 10, you can assume that the regression coefficients are poorly estimated due to multicollinearity.

Finally, either you can exclude the one or more factor with high multicolinearity or you can group in to one meaning full class for those are highly correlated variables by using either Principal component analysis(PCA) or Factor analysis (FA) to reduce the correlated variables from the model.

I hope this will be helpful to answer you question and minimize your challenge

.

Yemane wrote (emphasis added):

• However, for logistic we don't have that option. But we can solve this problem by using multiple linear regression for the set of independent factors excluding the original response and letting one of the factors as response and check the multicolinearity. Then use the variables with less multicolinearity as the independent factors for the original response.

No, do not replace the original dichotomous DV with one of the explanatory variables when using the REGRESSION.  Use the original DV, or use some other variable (even a randomly generated one) as the DV.  The multicollinearity diagnostics you get from the REGRESSION procedure have NOTHING to do with the dependent variable, and EVERYTHING to do with the relationships among the explanatory variables.  Therefore, the model estimated via REGRESSION must have exactly the same explanatory (and the same estimation sample) as your logistic regression model. 

To repeat, use exactly the same variables you have for your logistic regression when using the REGRESSION procedure, but pay attention to the multicollinearity diagnostics only from this model.  Do not pay attention to the table of regression coefficients or other output from the REGRESSION procedure.

HTH.

Very detailed and good suggestions @ Yemane & Bruce. Thanks

Very helpful answers.

run the logistic regression as a linear regression put one of the independent variables in the your model in the box as a dependent variable ( you can try each one of the independent variable in the dependent box and run the regression several times) you will have 2 important tables the first one is coefficient table ; in this table look for the following signs of multicollinearity:

1) VIF > 5 and if you wanna be more conservative VIF must be 0.20

the second table will be the Collinearity Diagnostics table; look for

a) Condition Index of 15 or higher

b) and at least two variables having Eigenvalue value of >.90.

hope that my answer helps. my reference is here bellow

Hair,J.F., Black, W.C., Babin, B.J., Anderson, R.E., Tatham, R.,L. (2006). Multivariate data analysis. Upper Saddle River, NJ: Pearson Prentice Hall.

@ Bruce Weaver ..

Dear Bruce,

First of all! Thanks for your contribution in answering my question. Can you please give me a reference in which someone has found multicollinearity in a way that you suggested? I think it may be very helpful.

I found this video is informative vif test multicollinearity spss through linear regression

https://www.youtube.com/watch?v=oPXjQCtyoG0

Thanks everyone for feedback, it helped me a lot

If you are using STATA, run the model (logit) and then type 'collin' e.g., collin x1 x2 x3 x4...

To check whether there is multicollinearity between variables perrtubation test is to be done .after doing that the features whose value changed a lot are treated as collinear in nature .

I met this problem, too. Thank you all.