if one dependent and independent variables more than one are available, multiple linear regression, Stepwise regression, and Ridge regression simply can be used. However, VIF values should be less than 10 for independent variables. if very high Pearson correlations between independent variables are found, Multicollinearity problem occurs in multiple linear regression, which is undesirable case. In multicollinearity problem, Ridge regression could be prefferred compared to Multiple regression. But, with the aim of definitely removing multicollinearity problem,

the better solution way instead of using multiple linear regression, Stepwise regression, and Ridge regression is to use Factor analysis or Principle component scores in multiple regression.

Thank you Ecevit

In my case ,am trying to find association between 6 SNPs (genetic polymorphisms) and a quantiative trait say VCDR (an eye parameter). And am trying all 3 different models additive, domiant and recessive. in few cases when i give all 6 SNPs as independent and VCDR as dependent the overall model is insignificant, but one or 2 SNPs is showing significance and the multicollinearity score (VIF) was in and around 1.3 thats the reason for my query whether to consider this significance as true significance.

Respected Ferdinamarie,

First of all, the Pearson correlations can be calculated simply.

if F value in ANOVA for multiple linear regression including all the independent variables was statistically insignificant, you must use STEPWISE regression (SR) to find statistically significant independent variables. SR can provide more flexible solution. In addition,

The simplest way is to use Simple linear regression between dependent variable and each independent variable.

Regards.

Stepwise regression is an appropriate way to eliminate the insignificant variables and then your full model becomes significant with some of few factors. But while you explain the factors, you have to explain the individual factors that which are significant and which are not significant. and also VIF should be calculated in order to check your multicoliniarity issue.

When sample size is enough, the best way to remove multicollinearity problem is to use factor analysis and principle component scores as independent variables in multiple linear regression. I would like to mention how to apply factor analysis and principle component scores as independent variables in multiple linear regression.

First, independent variables were exposed to factor and principle component analyses Second, suitable factor or principle component numbers should be determined for independent variables,Thirth, new-latent uncorrelated variables were derived from independent variables, Fourth, new-latent uncorrelated variables are used in multiple regression. In addition, Factor analysis must be specified with VARIMAX rotation

As a result, when VIF values for each new latent uncorrelated variables are calculated, thus multicollinearity problem are definitely removed.

Dear all: It seems that there is some thing missed.

The coefficient of determination (R2) indicates the goodness and fitness of the model. The higher the R2, the better the independent variable(s) explain(s) that the variation in the dependent variable. Small r indicates the significance of the relationships found. The main assumptions of regression are (Norusis, 1993, Berenson et.al, 2006 and SPSS 16.0, 2007):

1- Linearity test: States that the relation between variables is a linear relationship. To test the Linearity will depend on plotting of Studentized residual against the predicted value. When there is no relation between the predicted and residual values, in such case the model does not violate this assumption.

2- Independence of errors: States that the errors are independent from one another. Durbin-Watson test is used to test independence of errors, if D equals two, in such case the model does not violate this assumption.

3- Normality: Requires that the errors should be normally distributed at each value of X. To test the normality will depend on the histogram of residuals. If the shape follows the normal distribution, in such case the model does not violate this assumption.

4- Equal variance (homoscedasiticity): Requires that the errors are constant for all the value of independent variables. The equal variance is important for making inferences about βo and B1. To test the Equal variance will depend on the plot of Studentized residual against the predicted value. When there is no relation between the predicted and residual values, in such case the model does not violate this assumption.

5- Multi-Collinearity: Refers to a situation in which one or more variables are highly correlated. To test the collinearity, the VIF (Variance Inflation Factor) will be used. When VIF is less than 10, in such case the Collinearity model does not violate this assumption.

After achieving the mentioned above sex assumption of regression model, you can do Regressions.

In all analysis if r is significant then shows R square and F (ANOVA) will be significant. The second model shows which dependent variable is more effective (B and Beta) and more significant. if one variable is not significant, the problem can be found by factor component analysis (each factor should have loading value more than 40%), or reliability or normality test. These three should be done when you are setting the questionnaire through pilot study on 16-30 sample. If you did not do pilot study then you have to accept the variations.

Best regards for all.

Dear All

Thank you for your valuable suggestions.

Thanks to Abdel - Aziz for highlighting the assumptions of OLS. I think Ferdinamarie should take time off and read up the 10 assumptions of OLS to avoid violating them and running spurious regression. Apart from the significance of the coefficients, their sign identities (+or -) determine the direction of the interpretation. Positive coefficients show the existence of direct relationship and vice versa given that regression models try to ascertain the functional relationship between the dependent variable (regressand) and the independent variables (regressors). Also, watch out for the F-ratio which measures the explanatory power and overall significance of the model.

Thanks to Abdel Aziz for highlighting the OLS assumptions. Ferdinamarie should take time off and read up the 10 assumptions of OLS to avoid violating them and running spurious regression. However, apart from the significance of the coefficients, the ir sign identities determine the direction of the interpretation. Also watch out for the F-ratio which measures the explanatory power and the overall significance of the model

Ferdinamarie,

You have one outcome (dependent) variable, VCDR (vertical cup disc ratio), and six candidate predictor (independent) variables. VCDR is a continuous variable. Have you though of dichotomizing the outcome variable so that it is categorized as above or below a certain threshold? See British Journal of Ophthalmol. 2004 Jun;88(6):766-70. They looked at the feasibility of using the 97.5th percentile as a cutoff value. If you used this (or some other value) to classify your VCDR score into two categories, you could use logistic regression with VCDR category as the dependent variable and the six SNPs as candidate predictor variables. Using logistic regression would also let you analyze interactions, for example the effect of SNP2 X SNP5 on the outcome. This may be biologically appropriate because some genes interact with each other. There may be other established techniques for statistically analyzing genetic variables with a continuous or dichotomous outcome variable of interest, not sure what they are.

You can find a lot of things in our book

Dieter Rasch, Klaus D. Kubinger und Takuya Yanagida (2011)

Statistics in Psychology using R and SPSS

Wiley, New York

I hope that will help you