I have a nested sampling design with several transects within different catchments from two different regions. When performing analysis per region, I defined transects as my sampling units and, using GLMMs, used catchments to test for random effects. Since no detectable effect was found, can I just use GLMs from now on and mention in the text that we previously tested for random effects? Thank you in advance.
You can, but IMHO this is not a good idea.
A couple of reasons:
1) You had the asumption that catchments and regions can be different, and this assumption seems reasonable to me.
2) The random effects were not your study goal, so it is questionable why this should be tested at all.
3) Testing random effects to decide whether or not to include/exclude such effects from the model is actually inappropriate and leads to the same logical difficulties like all model selection based on data (via p-values, AIC or whatever). The only sensible criterion for model selection is "reasonability" basen on subject matter and the aims of the model (what should it be used for?).
4) Finally, a "non-significant" finding is no indication that the effect is absent and can therefore not be an argument to remove the random effect. I would accept this argument when the power is reasonably high (and I am quite sure that this is not the case) - but this would have required to think about what size of the random effect should be relevant and to design the study accordingly (and I am quite sure that this has not been done).
So to my opinion you must make up your mind what your model should be for. Since you already thought about such dependencies you seem deeming them significant (yes, yes), and so I'd find it consequent to keep them in the model. But if, however, your main focus is to have a model as simple as possible, even at the risk loosing the generalizability to other catchments and regions, well, then you surely can omit the random effect (but then I really wonder why you have considered them in the first place at all).
You can, but IMHO this is not a good idea.
A couple of reasons:
1) You had the asumption that catchments and regions can be different, and this assumption seems reasonable to me.
2) The random effects were not your study goal, so it is questionable why this should be tested at all.
3) Testing random effects to decide whether or not to include/exclude such effects from the model is actually inappropriate and leads to the same logical difficulties like all model selection based on data (via p-values, AIC or whatever). The only sensible criterion for model selection is "reasonability" basen on subject matter and the aims of the model (what should it be used for?).
4) Finally, a "non-significant" finding is no indication that the effect is absent and can therefore not be an argument to remove the random effect. I would accept this argument when the power is reasonably high (and I am quite sure that this is not the case) - but this would have required to think about what size of the random effect should be relevant and to design the study accordingly (and I am quite sure that this has not been done).
So to my opinion you must make up your mind what your model should be for. Since you already thought about such dependencies you seem deeming them significant (yes, yes), and so I'd find it consequent to keep them in the model. But if, however, your main focus is to have a model as simple as possible, even at the risk loosing the generalizability to other catchments and regions, well, then you surely can omit the random effect (but then I really wonder why you have considered them in the first place at all).
- Badges
- Science topic
- Similar topics
- Mathematical Sciences
- Statistics
- Statistical Modeling
- Analyses of Variance