Some collaborators and I have done an analysis using OLS multiple logistic regression, and adjusting the standard errors by multiplying by the square root of the design effect to account for clustering.
The data set has 2710 individuals clustered in 528 units (mean cluster size just over 5), but a lot of the clusters are singletons. One value of the dichotomous outcome variable has a prevalence rate of about 3%. Our most important predictor is a level 2 dichotomous variable (with approximately equal numbers of level 2 units for each value). We had an ICC of only 0.095, which according to Garson (2003) would not be statistically significant since the intercept was not statistically significant in the null model (see my other recent question), but one reviewer objected to the use of OLS rather than hierarchical methods because the main predictor was level 2 (and the most interesting results involved cross-level interactions).
Tossing out the singletons losses a lot of our data (and we might have to lose even more by tossing out other small clusters to use get good enough parameter estimates for a slopes-and-intercepts-as-outcomes model), and given the low prevalence rate of our outcome variable, I'm loathe to do that.
No, we can't collect more data -- this is a sample from a national database.
Any suggestions on a better way to handle the effects of clustering in this situation (that can be carried out using SPSS and/or HLM 7, the two packages we have and have facility using) would be much appreciated.
Please check the following:
Article Estimating Multilevel Logistic Regression Models When the Nu...
https://www.researchgate.net/post/What_would_be_the_most_appropriate_ways_to_analyse_hierarchical_data_with_dichotomous_outcome_one_outcome_rare_and_many_singleton_clusters?isAnswerFieldFocused=true
Thank you. I'd seen that article, but our problem is the opposite: we have 528 clusters, but many of them have only one member.
Dear David,
Just some thoughts, and you may have already considered these things, but:
1) why logistic regression? the general topic to look up might be quantile regression - where there are. number of methods available for relatively rare events.
2) HLM7 should, if told about a Bernoulli outcome, do the right thing with small cluster sizes. Then rerun the model with no clusters (just create a lvl 2 ID variable). This lets you put a number on any clustering effects - and possibly make a principled argument against their inclusion. After all, it is not the p value of the ICC so much as the estimation of any biases introduced by ignoring it.
3) I might personally be tempted to use a permutation test with with cluster resampling - then you can keep your h-model as requested
4) see Article Evaluating Models for Partially Clustered Designs
Good luck.
- Badges
- Science topic
- Similar topics
- Mathematics
- Statistics
- Sampling Design
- Sample Size