Hi don't panic just yet you have some errors in your understanding of the least squares requirements. First download a copy of Kutner et al Applied linear statistical models 5th ed from the z-library. Then get scatter plots of the DV against each IV and see what they look like. Now linear here means linear in the regression coefficients not in the IVs so you should be good there. Now go through Kutner et al and check everything you did and if you skipped something do it now. If you are still worried at this point then please ask another question. Best wishes David Booth

Dear David Eugene Booth ,

I have scatterplots for the DV against each IV. These all do not show linearity. I have followed the instruction very carefully. So, my question remains the same; How do I proceed when the assumption of linearity is violated in a multiple regression?

Kind regards,

Stef

Can you show us your results?

The Consequences of Violating Linear Regression Assumptions | by Hajime Alabanza | Towards Data Science

here is a good answer.

https://www.linkedin.com/in/halabanza/ | [email protected]

Dear Rainer Duesing Rainer Duesing ,

I am a bit secure with not giving away too much information for it is sensitive data. I will add the scatterplots with total fit lines, but without descriptives of what is measured. I can say the x-axis is a 7 point likert scale and the y-axis is a total score of a questionnaire.

Sadly I have just noticed that the independence of observations has been violated too, as assessed by a Durbin-Watson of 0.073. This means I can not continue with a multiple regression anyway...

I am still curious to what you make of the scatterplots! And if you have any recommendation to what multivariate analyses I should look for now? I have to include that in my thesis... Would a Spearman correlation with a covariate be an idea?

Kind regards,

Stef

Stef Molenaar could you please elaborate how you concluded that linearity was violated? I cannot see any obvious problems with the residuals.

The Durbin Watson test only makes sense, if you have a suspicion that the data are dependent in the data generating process. This also means that the order of the data in your data sheet is fixed and meaningful. If you have collected random participants, there is no good argument per se that there should be an autocorrelation. Your output looks like you use SPSS, just try this: order your data in the data viewer according to the participants ID (or time they attended), run the regression and observe the DW-value. Now, order the data according to any other variable, re-run the regression and observe the DW. It should be different, since the ordering has to be meaningful for this analysis. This totally makes sense, since autocorrelation means that past events "inform" further events, therefore the data has to be in that particular order. If participant do not have any contact to each other or are able to comunicate, "inform" each other, autocorrelation is not an issue.

Rainer Duesing I concluded that linearity was violated, because the second and fourth scatterplot are near-perfectly horizontal. I realise now that this is still a linear relationship...

Furthermore, I had to check for the assumption of independence of observation via DW too. I believe I had to do this and it can't be violated, because it is an assumption for multiple regression, atleast according to my source (LAERD Statistic guides). Your reasoning for why it may not be impactful for my analysis makes sense and I will definitely remember that! Do you happen to have any further source, so I can read into it and cite it in my reasoning for possible ignoring that violation?

I apologize for my lack of knowledge on this subject or if I fail to specify what you ask for. This is my first attempt at an scientific article including data-analysis. Much to learn still!

@Sef Forget linearity it means linear in the regression coefficients. Please see KUTNER ET AL APPLIED LINEAR STATISTICAL MODELS 5th ed, p,221, while you are at it please read the rest of the book because you are very confused about linear statistical models. David Booth

David Eugene Booth Fair enough. Thanks.

The observations don't necessarily need to be independent. It's fine to not know what to do: ask someone in real life to help you out with the data analysis part

Stef Molenaar I think there are two excellent and seminal books about regression in general, which you should consider, both have information about non-independence. Cohen, Cohen, West & Aiken (2003, ch 4.4.5, pp. 134) for example explain different kinds of dependence, from which autorcorrelation ist just one type. Another form may be clustering, for which a multilevel model may more appropriate than OLS regression. All depends on your study design and I cannot say anything concrete without knowing your data and data generating process. Another good source is Darlington & Hayes (2016, ch 16.2.5, pp. 506) with other expamples how dependence may occur.

Generally, non-independence should not bias your parameter estimation, but may affect the standard errors in both ways (too large or too small SE) so that your inferential statistics are inappropriate.

Cohen, P., West, S. G., & Aiken, L. S. (2003). Applied multiple regression/correlation analysis for the behavioral sciences, 3rd ed. Psychology press.

Darlington, R. B., & Hayes, A. F. (2016). Regression analysis and linear models: Concepts, applications, and implementation. Guilford Publications.

Stef Molenaar -

I think you should be careful about copyright for a 'free' book. There is a great deal available on the internet from the Pennsylvania State University online courses that they make available by using a creative commons license, I was told in conversation with a professor there. At any rate, I suggest you research the terms "graphical residual analysis," and "cross-validation" online. I would be skeptical about downloading a book or article that may be pirated.

A graphical residual analysis will show how well your model fits your particular sample. Cross-validation is to try to avoid overfitting to one particular sample so closely that your model does not fit other parts of the population well where you may want it to be useful.

For heteroscedasticity you may find this useful:

https://www.researchgate.net/publication/333642828_Estimating_the_Coefficient_of_Heteroscedasticity

and

https://www.researchgate.net/project/OLS-Regression-Should-Not-Be-a-Default-for-WLS-Regression, and the various updates and references there. 

Regarding autocorrelation, that is usually with a time series. Rainer Duesing gives good advice on that several entries above.

Best wishes - Jim