Hello, I'm a student research looking to apply a multiple linear regression to my data set of all continuous values. My question is mainly about a step-by-step idea of how to do this, as I believe I understand the general gist of it all, but not quite the whole picture.
To start, it is my understanding you plot a scatterplot of your independent vs dependent data and your independent vs independent data. This helps determines if there's a linear relationship between the variables, and if there's any collinearity between your independent variables. In doing so, after this step I can eliminate the independent variables that either have a strong collinearity or no relationship with the dependent variable.
After the initial screen, I am under the impression I can run a step-wise(or forward/backward) multiple linear regression slowly adding in the variables that fits the BEST MODEL. Is that a correct way to look at it?
Doing a meaningful analysis is, unfortunately, not that simple.
The problem are possible interactions between variables that may either obscure non-linear relationships with the response variable or wrongly imply a non-linear relationship if the impact of the other variables is not considered at the same time (i.e. when you look at the marginal relationships). There is no easy way out. You need to make some assumptions, and they should be resonable - and justifying this is not done by means of statistics but by expert knowledge of the subject matter. A help - to a certain limited amount - can be residual diagnostics: if the residuals do not show any pattern and seem to be a sample from a normal distribution, then this is a slight hint that the model might be approriate or at least that it does not miss anything really important (like an important interaction or non-linear relationship).
Stepwise variable selection is a very bad approach to build a model. You should definitely avoid this. The selection of variables (and non-linear relationships, and interactions) should be based on theoretical considerations, expert knowledge of the subject matter and the purpose of the model. If you consider a variable important enough to inlcude it in the model, then there should not be a statistical reason (like the lack of a statistically significant effect estimate given your data) to exclude it from the model.
If you have many variables, and you are just looking if they allow you to somehow predict the outcome, modelling using linear models (or alikes) may actually not be the best choice. You may consider regression trees, random forrests or deep learning approaches. It's evident that this needs quite a lot of data, but this is the proce you have to pay if you have no theoretical concept but still want to create/build some tool that uses data to make good predictions.
And, finally, there is no BEST MODEL. There are many different possible models, all with their own pros and cons. Only if you have very, very specific demands what the model should do (and if you are able to formulate this in a mathematical way) you may be able to identify a specific model that is best suited for exactly this purpose. And again, decinding on this best model is not (only) a matter of statistics you calculate for your observed data - it remains mainly a task requiring expert knowledge in the subject matter.
If you are expert in the subject matter, it may be very helpful to cooperate with a local statistician to identify a good solution.
I believe that the chapter on linear regression in Andy Field's book (Discovering Statistics using... [Insert software's name]) is what you're looking for. This should give you a step-by-step understanding of the basics of multiple regression. After reading it, you should be better able to appreciate the important comments made by Jochen Wilhelm in a previous answer. Good luck!
Here's a helpful hint don't look at anything by Andy Field. His programs may run but his statistics is mostly just wrong. Best wishes, David Booth
- Badges
- Science topic
- Similar topics
- Psychology