I have collected data on responses to a questionnaire but have found ceiling effects for three questions. I wonder at which point it becomes pointless running a GLMM. Is there a hard and fast rule for this e.g. 10%, 10% and 80%?
The question I have with the worst skew has a distribution of 4, 22 and 139 for the three different options and the one with the least concerning skew has 12, 55 and 93.
Any comments will be greatly appreciated.
I assume you mean Generalised Linear Mixed Model, right? Hard to tell from the information you give about your outcome... I am not aware of a rule, but would probably go with a dichotomisation if you have a 'face valid' and sensible explanation why you assign the different response options to 0 and 1. You might also consider to transform your outcome (log?), but if you have e.g. just 4 response options this won't help much. Also you can compare the different ways to handle the outcome variable (dichotomised/logistic, normal or transformed), if substantive interpretation doesn't change it really doesn't matter much...
I'm also not aware of a hard-and-fast rule. A few options I can think of off the top of my head:
- Bootstrapping (https://thepsychologist.bps.org.uk/volume-22/edition-5/methods-giving-your-data-bootstrap). This should be relatively robust to weird distributions. If you're using R, there are functions implemented for this (like bootMer), and it's relatively easy to write your own code for it too. Unfortunately, bootstrapping requires running several hundred models, so given that a glmm is already pretty slow, this could take a while (I often just need to let it run overnight).
- Throwing out those items. That is probably not a very attractive option, but I guess it's worthwhile to at least think about the possibility that, if these items are at ceiling, they're maybe not giving you much information or are not working as you expected them to.
- Transformation. Like Michael mentioned, this might help out... although with really extreme skew it still won't (e.g. if you use a 7-point scale and 75% of your data points are 1, no transformation is going to change that), and there is debate in general over whether transformation is a good idea.
- Changing the nature of the variable. This is also like Michael suggested. For example, if 2/3 of your data points are at 100% on some measure, then rather than treating it as a continuous measure it might be better to treat it as a binary (100% vs. anything else).
Thank you both for your advice. I think I shall try changing the variable to binary and hopefully this should improve things.
I would hesitate to dichotomize the variable, because you are losing a lot of information by doing so. Not that it isn't sometimes necessary, but there may be other alternatives. Really, it depends a lot on exactly what type of GLMM you are running ... GLMMs are a pretty broad class of models. It's hard to recommend an approach with more information.
What is your data? You say it is responses to a questionairre, but you seem to be implying that the outcome is continuous? What exactly is the outcome variable? Which distribution/link function did you choose for the GLMM? A normal distribution with an identity link won't handle skewness very well, but a negative binomial distribution with a log link WILL (as long as the outcome is a positive integer), etc.
- Badges
- Science topic