Attached a paper regarding Rules of Thumb and some link in ResearchGate:

https://www.researchgate.net/post/Multicollinearity_issues_is_a_value_less_than_10_acceptable_for_VIF

https://www.researchgate.net/post/How_high_of_VIF_value_in_regression_can_be_accepted

In regression analysis you read the tolerance (must be less than 10) and VIF (must be more than 10) values. they are each other's opposites. Multicollinearity means that the independent variables are correlated at a level at r= .90). We must not ignoire the values because the outcome will then be distorted.

It is good and fundamental question of data analysis. In practice the raw data may have very high linear dependency (or collinearity ) or noise or mutual nesting of data points ...or similarities (All these are almost similar concepts), and this is why we computing errors in data. For the ideal cases the data may be linearly independent or in other words orthonormal & orthogonal with probability of its sum equal to one, however., it can be happen for very very clean data. In order to get good accuracy, we therefor perform some data cleaning methods (before data processing via proposed model/function/algorithm/method etc.,), e.g., SPA/SVD, Wavelet methods, local discretization, Kalman Filtering and Grassmann Filtering etc.

You can look into the following book for a thorough discussion on issues with multicollinearity and methodologies for estimation in presence of Multicollinearity:

" Mathematical Analysis of Spectral Orthogonality "

by Kalivas and Lang

I just want literature references for this choice of variables. can you please give me some journal or book references?

Dear Kaushik Ghosal ,

Andy Field's book on Discovering Statistics using SPSS is a good source, so is Tabachnick and Fidell's book on using Multivariate Statistics. However, whilst bivariate correlations, VIF and eigenvalues can help, the choice of which variable to exclude is also a qualitative decision, rather than just a quantitative one.

Dear Kaushik,

Since the independent variables are nonstochastic, you cannot test for multicollinearity. You have to find some practical means to get around the problem. If you know that two variables are colinear, for instance if you are are using weekly and monthly data for the same variable, then you can drop one of them. That is like a first check.

You can also find out if two independent variables are correlated by regressing one on the other. The residual of such a regression will be the uncorrelated part of the dependent variable. You can then use that residual to stand for that variable in original equation.

This kind of insight comes from practice. For instance, P. Kennedy book on "A Guide to Econometrics" explains such two variable equation with Venn Diagrams. You can see that the non-intersection part of the dependent variable is purged of colinearity. Also, that book shows similarly how the errors of estimates will affect your test statistics if you do not correct for multicollinearity.

Good luck in your research

Thank you Lall B. Ramrattan , Peter Samuels

Attached a paper regarding Rules of Thumb and some link in ResearchGate:

https://www.researchgate.net/post/Multicollinearity_issues_is_a_value_less_than_10_acceptable_for_VIF

https://www.researchgate.net/post/How_high_of_VIF_value_in_regression_can_be_accepted

Thank you Roberto Molteni

You may find this blog post from Paul Allison on when you are safely ignore multicollinearity useful: https://statisticalhorizons.com/multicollinearity

Thank you Stephen Parry

Personally, I do not think ignoring multicolinearity is the best thing to do. It is a situation that occurs naturally. One, therefore, has to first acknowledge it, then find a way around it rather than ignore it as if it didn't exist.

Ignoring MC is not an option at all. It can easily be shown that the standard errors of the OLS estimators will be inflated if there is strong (severe) MC. Consequently, the t-ratios will be 'small' leading to incorrectly failing to reject the null of zero coefficients (Type II error). The most commonly used remedy is dropping co-linear explanatory variable(s) based on, for instance, the variance inflation factor (VIF). Dropping variables may of course result in model misspecification (but the lesser of two evils). Another possibility is estimating the model using ordinary ridge regression. But this approach is criticized for being 'atheoretical' (not driven by theory).

Multicollinearity must not be ignored. To do so biases your results and can give misleading confidence to single variables. If you include interaction terms you can get a more accurate estimates of the effects of coefficients for each single explanatory variable and better understand the extent to which two (or more) variables explains the same phenomena,

You can find the topic dealt with in some standard text-books. For example, Chapter 16 of "Applied Regression Analysis" by Draper and Smith (3rrd Edition) 1998, Wiley.

Thanks, David A. Jones Patrice Showers Corneli

Dear Kaushik Ghosal

Multicollinearity (MC) is when a set of independent variables have strong correlation between them.

Depending on what you are looking for understand phenomenon, MC will help or mess up. Multivariate Regression model hates and Factorial Analysis (Principal Components Analysis) loves it.

I like to use the example of valuation of used cars for sale in a regression model to explain effects of MC.

You believe that the sale value of a car depends of some factors like: size, power of engine, fuel consumption, weight, odometer value, tire status, paint quality, age.

You sample many cars in the market in order to build a model to rationalize the valuation. A Multivariate Regression Model.

Clearly you can note that you have 2 very significant groups of variables. They have, inside each group, high correlation.

Group 1: size, power of engine, fuel consumption, weigh are very correlated because large car has bigger motor that drinks more fuel and needs large chassis.

Group 2: odometer value, tire status, paint quality and age also are very correlated and reflects age/use.

Group 1 and 2 don’t have correlation or it is small.

MC in this model is very significant and the problem is, mathematic when define the coefficients of the variables in the regression model keeps confused to attribute the enough relevance in order to “push” the variance of dependent variable. At the end, is like to share all the influence to more than one independent variable. And you really don’t know what is going on. Crazy model ! Which way (intensity) the variance of a specific independent variable is affecting the Variance of dependent variable.

How to minimize this effect? Smart selection of variables, basic correlation analysis (or better, Multicollinearity analyse when you have software to help you. I use SPSS and works fine). Sometimes it hurts because you will sacrifice some variables you really believe would contribute most to reproduce the phenomenon in your model.

Keep in mind: Good quality regression model has high correlation between dependent and independent variables AND low correlation between pairs of independent variable.

But how much MC would be acceptable? The same criteria of keep or remove outlier, because eliminating some colateral effects in a sample will result in model overfitting and the model seems to be unreal because is simply perfect just for the present sample and you want to estimate the behavior of the population.

you should consider multicollinearity to have a meaningful model. you must not ignore it.

Babak Jamshidi can you please give some book or journal references?

Kaushik Ghosal Generally, The books under the title of Advanced Regression Analysis, Linear models, Advanced Linear Models include this topic. For example:

Regression Diagnostics - Identifying Influential Data and Sources of Collinearity

Wiley-InterscienceDavid A. Belsley, Edwin Kuh, Roy E. Welsch

And

Regression diagnostics: Identifying influential data and sources of collinearity

WileyBelsley D.A., Kuh E., Welsch R.

Good Luck