Hi everyone! I'm fairly new to regression modelling, and was hoping someone could help with what I hope is a very basic question.
I have a set of repeated observations on a regular grid (i.e. they vary by: latitude, longitude, and time). Of these observations I have a dependent variable, y, which I would like to model as a function of an independent variable, x. From literature it is known that x has a significant negative effect on y, but the magnitude of this effect is often mitigated by other variables which also need to be accounted for. A lot of the variables I have to choose from are also quite correlated with each other, as well as with y.
So far, I've tried using a backward stepwise regression with ordinary least squares: I start off fitting all the variables I have against y, and then remove variables until the p-value for all remaining variables is < 0.05. If needed, I then remove variables until the variance inflation factor (VIF) for all remaining variables is < 10.
While this produces models that have a fairly high R^2, I find that often either x has the wrong sign, or was removed by the stepwise process entirely. I am sure that my approach is too simplistic, as in some areas my variables do not appear to linearly correlate with y. I've tried getting around this by fitting separate models for different regions, but this does not help.
I understand that longitudinal data requires at least a linear mixed effects model, but am unsure as to how to set one up for my dataset from the examples I've seen online. I've also read papers where tools like LASSO are used to select the best parameters, but these also require some pre-conditioning of the data which I am also unfamiliar with.
I would be most grateful for any advice anyone here could give me.
Because of the repeated nature of the data, you need a model that accounts for the dependence over time. It may also need to take account of the dependence over space.
Starting with repeated over time have a look a this material
http://www.bristol.ac.uk/cmm/software/mlwin/mlwin-resources.html#multipleresp
here is a straightforward example
http://www.bristol.ac.uk/media-library/sites/cmm/migrated/documents/repeated-measures.pdf
I second Jochen's guidance about p hacking.
The final chapter of this considers space time models :
Book Developing multilevel models for analysing contextuality, he...
your question is unusually coherent, well done.
and I need to know about the longitudinal element, how many repeats do you have? And is there any missingness.
Thanks! I tried my best to be as thorough as possible, because I wasn't sure how unusual my case was.
For each lat/long grid cell I have 13 repeats (annual values). There are no missing values in any of the variables I have.
Have you considered spatial regression models?
Be careful with stepwise procedures. Youu shouldNOT exclude a variable because it failed to get a "small enough p-value". The decision which variables a model should have should be made based on theory. And it is usually better to adjust for variables that are demeed relevant (even if the data at hand does not give a small p-value for it).
Removing variables with a high VIFs seems to be ok for me, as they indicate a relevant misspecification of the model. It may be interesting to understand why some variables have such large VIFs.
Because of the repeated nature of the data, you need a model that accounts for the dependence over time. It may also need to take account of the dependence over space.
Starting with repeated over time have a look a this material
http://www.bristol.ac.uk/cmm/software/mlwin/mlwin-resources.html#multipleresp
here is a straightforward example
http://www.bristol.ac.uk/media-library/sites/cmm/migrated/documents/repeated-measures.pdf
I second Jochen's guidance about p hacking.
The final chapter of this considers space time models :
Book Developing multilevel models for analysing contextuality, he...
Thank you both for your answers! I will review the literature and try improving my models with your advice.
- Badges
- Science topic
- Similar topics
- Mathematics
- Statistics