Dear
I have an outcome (dependent) variable (scores received from a multiple-choice test/computerized reading comprehension test), and two predictors (independent variables) (the data received from two Likert-Scale questionnaires/Computer Familiarity Scale and Attitudes towards Computer Scale were transformed to continuous scales). I'm trying to probe the directionality of the effects i.e the effect of two independent variables on the dependent one. Moreover, the correlation among the variables should be assessed based on the results of coefficients determination (R2). I came to the solid conclusion that the Multiple Linear Regression may be the best statistical test to examine my research hypotheses. What’s your idea? is it the most appropriate test to run based on the aforementioned data? If yes, which assumptions should be met for running Multiple Linear Regression statistical test? Somebody says that the results of just two Normality and Durbin-Watson (absen of autocorrelation) are to be reported, and this is the accepted procedure for statisticians. But it seems that those two are not enough, and totally, 8 main assumptions should be met in this case, and the most important ones are .........
1. Linear relationship 2. Multivariate normality 3. No or little multicollinearity 4. No auto-correlation 5. Homoscedasticity.
Would you please inform me if I am required to meet all the assumptions for running Multiple Linear Regression test? or Not?
It is worth mentioning that I'm supposed to report the most appropriate, accurate and subtle results of statistical procedure in order to get the positive judgment of austere reviewers!
Thanks in advance
Warm Regards
Seyyed,
In a nutshell, here are the assumptions required to run MLR: (i) linearity of the independent variables with the dependent variable y ~ f(x1,x2, ..), (ii) independence of errors (no dependency among the independent variables), (iii) homoscedasticity or equal variance of errors across all levels of the independent variables, (iv) collinearity meaning the independnt variables are not correlated and finally (v) normality i.e. that the variables or errors are normally distributed.
Thanks for your constructive comments.
Actually I examined the normality of residuals by Kolmogorov-Smirnov test and checking normal P-P plot of regression standardized residuals. Then, the homoscedasticity was also examined by checking Scatterplot, and no tight or wide distribution of points was shown on both sides of the plot. In fact, two normal distribution and homoscedasticity assumptions were satisfied. Could the satisfaction of Linearity assumption be taken for granted if the aforementioned assumptions are met? Or I'm required to check the Linearity separately?
Then, absence of multicollinearity (the correlation between independent variables) was confirmed by the Variance inflation factor (VIF) value.
Finally, the 2.08 Durbin-Watson value confirmed that there was no autocorrelation (neither positive nor negative autocorrelation) between variables. That's all what I did.
I'll appreciate if you inform me of the appropriateness and accuracy of my procedure. Is there anything else I should do?
Regards
Hi Sir,
Multiple linear regression analysis makes several key assumptions:
- linear relationship
- Multivariate Normality
- No Multicollinearity
- Homoscedasticity
Would you please tell me if the relationship between the two DV and IV is linear based on the Scatterplot and points distribution?
What about this ?
Actually, I have moments of doubt to confirm the linear relationship between these two DV and IV due to the cone shape of the points distribution? I can't confirm the linearity based on this scatterplot! Are you in agreement? or have other ideas?
From your description, linear regression could be a useful tool. It is not really a test, but it has some tests associated with it. You are correct that there are a number of assumptions associated with linear regression, but whether you need to satisfy all of them depends on how you plan to use linear regression. A quick review these assumptions is as follows. More information (and software to carry out the various tests) can be found at www.real-statistics.com.
1. Linearity – You have two independent variables and so you should create two scatter charts: (1) independent variable 1 with the dependent variable and (2) independent variable 2 with the dependent variable. The data on each of these plots should align reasonably well with a straight line (i.e. match the linear trendline in Excel)
2. Normality – The residuals (the y data values minus the y values predicted by the regression model) should be normally distributed. You can check this by using the Shapiro-Wilk test (or QQ plots, etc.)
3. Other assumptions about the residuals – The residuals should be randomly distributed with mean close to zero. You can test this graphically or by using Cook’s D (or other similar tests)
4. No Multicollinearity – You won’t be able to get a linear regression model if there is exact multicollinearity. You can remove multicollinearity by removing one (possibly more) of the variables that is causing multicollinearity. Exact multicollinearity is not common with real data, but you can have a high level of multicollinearity (especially if you have a lot of independent variables). You can detect the possibility of high multicollinearity if the VIF values for some of the variables are high. The problem with high multicollinearity is that the standard errors of the regression coefficients will be high and so the predictions from the linear regression model won’t be very precise. You can reduce the impact of multicollinearity by using Ridge regression or some other similar method.
5. Homoscedasticity (homogeneity of variances) – When you graph the residuals against any of the independent variables, you should see a random pattern. If the resulting scatter chart looks like a line, a regular curve or any other pattern that isn’t random, then you have the possibility of violating this assumption. You can also use the Breusch-Pagan test. You can address violations of this requirement by using a transformation of the data or a correction to the standard errors of the regression coefficients (what are called robust standard errors).
6.No Autocorrelation – You can use Durbin-Watson to detect first-order autocorrelation. This test won’t detect other forms of autocorrelation. The Breusch-Godfrey test can be useful in this case. Autocorrelation tends to be an issue with time series data since the data in one period (year, month, etc.) tends to be correlated with the data in the previous period (first-order autocorrelation) or possibly an earlier period (p-order autocorrelation). Autocorrelation can be addressed using techniques such as Newey-West standard errors.
7. The sample size should be larger than the number of independent variables (otherwise the regression model won’t be defined — k linear equations in more than k unknowns).
Unless assumption 7 is violated you will be able to build a linear regression model, but you may not be able to gain some of the advantages of the model if some of these other assumptions are not met. Normality and Durbin-Watson (actually Autocorrelation is the assumption) are not the only assumptions that are important. In fact, for large samples it tends to be less critical to check for normality since the Central Limit Theorem will kick in).
Charles
- Badges
- Science topic
- Similar topics
- Vectorization