And also if the tolerance is 0 and there is no VIF, does it mean a violation of the assumption? Am I right or wrong? If there is a violation, what do I do?
There are many reasons you can get a R^2 value that is "low". If you are running a Multiple Linear Regression (MLR) you need to look at your residuals. They need to be normally distributed and not show any patterns. The common patterns you see are fanning and rainbows. Each of these tells you that you need to modify your model. The residuals need to be independent from each other too. So, taking a monthly health assessment from the same person 2- years before and 2-years after a surgery is not proper. (This is called panel data/repeated measures.)
When it comes to VIF, the ideal VIF is 1.00. If your VIF is above 5.00 or 10.00, depending upon your source, you have issues with your data. Some of your independent variables are correlated with each other. This can lead to a really bad model. For instance, one data set I looked at had a several terms with a high VIF. The model clearly showed that smoking decreased the chance of getting cancer and heart disease. When I removed the terms with high VIF, I found that smoking is bad for you. The diagnostics on the second model showed that the second model was better.
How many factors do you want to test? What type of model will you make?
Multiple regression is similar in concept to regular regression, but instead of trying to minimize the distance from Y value to the best fit line, i it is attempting to minimize the distance from a plane in n dimensional space to each Y value.
As far as VIF (1/(1-R2)) and tolerance (1-R2), they are interrelated and therefore low VIF's should show acceptable tolerance values. A VIF of 10 would assume a correlation between the two variables of .95 (1/(1-.948^2)) and a tolerance of 0.1. With the VIT cutoff values of 5 you are looking at correlations of approximately .90 and at the more strenuous VIF cutoff of 3 you are seeing a correlation of approximately .8.
If you have items that are highly correlated, you have a few remedies. You can drop one of the items, mean center the variables, standardize the IV or use a different tyoe if regression analysis (ridge or principal component).
Explanations above are good. I'd add one thing. You could estimate the variance of the prediction error. You might want to consult econometrics texts such as those by GS Maddala. The link attached here is for a flexible method of handling multiple regression for a given environment where totals of predicted values are needed, but you can see in there what is relevant to a single point, and some references. With variances you can construct confidence intervals. This could be used to compare models.
Article Using Prediction-Oriented Software for Survey Estimation
Hello all,
I don't know if the OP above was able to find a solution to this peculiar problem. I have a similar problem in that, instead of having a VIF above 5/10, I have a tolerance=. and VIF=. I have no idea why my tolerance values and VIF would be MISSING. Any ideas of what might be responsible for this and how I can rectify it? Thanks in advance
- Badges
- Science topic
- Similar topics
- Medicine
- Physiology
- Biosignals
- Biomedical Signal Processing
- Statistical Analysis