Hi all
This is a question posed out of curiosity more than anything else, but I'd appreciate any answers that helped me confirm my hunch it's a bad idea.
I have a multilevel dataset with individuals (n=600) nested with neighbourhoods (n=248) nested within counties (n=30). Because there are too few individuals per neighbourhood (mean = 2.4), it is not possible to consider this level in a ML analysis because there are too few individuals per neighbourhood to derive valid inferences. However, I can fit a (less-than-ideal) model of individuals within counties where there are sufficient numbers per county to permit valid inferences, although the data are unbalanced.
My question is this, if I fit a random intercepts model on my outcome, with individuals nested within counties, can I still include fixed-effects of environmental variables at the "neighbourhood-level"? I think the standard errors for any effect sizes will be too wide, but what other reasons might there be affecting the validity of interpreting fixed effects from a level not specified in the random effects part of the equation? The fixed effects at this level would provide a more accurate characterisation of individuals' exposure to environmental characteristics than the equivalent variables derived (and averaged) over the entire county.
Thanks in advance for helping me understand this issue. Any directions to reading on this topic are gratefully received.
Thanks
James
Ignoring a level of clustering based on sparse clustering typically is going to introduce a fair amount of bias into the inferential tests of the regression coefficients because there's variance not being modeled and it'll be haphazardly distributed to various other parts of the model. Natasha Beretvas and her colleagues have written a few papers on issues of ignoring levels for various reasons with cross-classified models that shows this.
On another note, have you considered using design-based methods to account for clustering instead of multilevel models? My apologies for plugging my own work, but I had a paper in Psyc Methods last year showing how design-based methods (GEE, cluster-robust errors) are not greatly affected with sparse clustering. If your outcome is continuous, the interpretation of the coefficients will also be identical to those from a multilevel model.
Ignoring the clustering will introduce bias as you will be violating the assumption of independence of errors. However, this can be an issue regardless of whether you are using a predictor at the neighborhood level or not.
On the other hand, specifying random effects with small number of participants per neighborhood can also potentially result in biases. However this is actually mitigated by the fact that you have a large number of groups (e.g. Theall, et al 2010, Impact of small group size on neighbourhood influences in multilevel models. Journal of epidemiology and community health). From this perspective I think that it is preferable to use a random intercept for neighborhoods with small groups rather than ignore it completely. This is regardless of whether you will end up using the environmental variables in your analysis.
As Daniel suggested, you may also want to consider alternative ways to test your model. My preference would be for a Bayesian hierarchical model because it will allow you to test the exact model you want and small groups would not be an issue. (uncertainty about the estimates due to small groups will be reflected in the results via wider credible intervals).
Daniel and George
Many thanks to you both for your helpful and illuminating answers. I will investigate these various methods. I have some experience with HBM and this is a good idea, so I may follow this one up immediately. Daniel; I'll also have a read of your paper which seems highly relevant. Thanks both.
James
I would suggest modeling the full hierarchical data structure using maximum likelihood estimation with Gaussian quadrature (>=7 points). This method would give you accurate estimation. You can look at the simulation study in this paper: Elghafghuf et al 2014. Survival analysis of clinical mastitis data using a nested frailty Cox model fit as a Poisson mixed-effects model.
Even if you have few individuals but plenty of neighborhoods I would keep neighborhood as a level so that if there is any between neighborhood dependency the multilevel standard errors of the fixed coefficient of the estimated effect of a measured neighborhood variable take this dependency into account.
I think that there is a lot of confusion about sample size due to the issue of target of inference - this is what I wrote in relation to a previous question
"A lot depends on your target of inference. Taking the example of pupils (level 1) in schools (level 2) - if you want to infer about the between -school variance than you need plenty of schools. If you want to also infer about a particular school you also need plenty of pupils in that school. This issue most obviously comes up with people in households - you can do a multilevel analysis on this if you have plenty of households but you are unlikely to be be able to say anything about any specific household as the number of members is likely to be too low for precise inference. In a similar manner multilevel models can be used to analyze data for twins where there can only be 2 people (level 1) in the twin pair (level 2).
At the other end of the scale where you have thousands of people (at level 1) in a country (at level 2)- a simulation study has shown ( and using MCMC Bayesian methods) that the method works quite well with only a dozen or so countries even when there is a discrete outcome and you are estimating cross-level interactions
How Many Countries for Multilevel Modeling? A Comparison of Frequentist and Bayesian Approaches http://onlinelibrary.wiley.com/doi/10.1111/ajps.12001/abstract
For general orientation on this - see
http://www.bristol.ac.uk/cmm/software/mlwin/mlwin-resources.html#datadesign
"
Back to your question - I would not drop the level but specify it and if it comes out to be very low or indeed zero, so be it.
- Badges
- Science topic
- Similar topics
- Geoscience
- Cartography
- Mapping