Hi all,
I am doing a moderated mediation regression in process (model 7). I have covariates with a lot of levels, (e.g., 5) and created dummies.
Now, when I run the PROCESS model, I get the error: NOTE: Due to estimation problems, some bootstrap samples had to be replaced. The number of times this happened was:2740.
When I run the model with the categorical variables (no dummies), I do not get the error, but then I do not have dummies. What would be the most simple option for now?
Thanks in advance.
Kind regards,
Maayke
How large are the groups you are comparing with each dummy variable?
Dear Maayke Mijnsbergen
These links might be helpful, have a look:
1. https://ncss-wpengine.netdna-ssl.com/wp-content/themes/ncss/pdf/Procedures/NCSS/Mediation_Analysis.pdf
2. https://www.youtube.com/watch?v=XY_NB-xJ5Mg
Kind Regards
Qamar Ul Islam
Rainer Duesing Thank you for your answer. My groups are 91 and 99 (in total 190).
Qamar Ul Islam Thank you for your contribution! I will take a look at the documents.
That's one known problem with the bootstrap. To me it's not surprising. Some resamples are not estimable so you get an error. Using weights in the regression is a solution to this
Stefano Nembrini can you elaborate under what circumstances this is likely to occur?
Maayke Mijnsbergen this cannot be. You say that you dummy coded a 5-level variable, which will lead to 4 dummy variables. How was the sample size in each of these variables?
Rainer Duesing Hi Rainer, yes, I used the four dummies in my analysis, not the five dummies. But I created 5. And did not include the baseline.
Yes, it is clear that you use 4 dummy variables if you have 5 groups. But in case of dummy coding (I simply assume this coding scheme) you have one reference group and 4 comparison groups. How large were the sample sizes for each group?
What do you mean with "And did not include the baseline"?
Rainer Duesing it is actually connected to what you asked concerning the size of the groups: I think Maayke Mijnsbergen means that the reference group was not included in the model [but those two things are equivalent]
When bootstrapping is implemented as sampling with replacement of the rows [also called x-y bootstrap], with the same model being used on different [bootstrap] datasets, dummy variables become problematic and even more so when the dependent variable is more discrete [or 0-1]
if variable X1 has 5 categories but some of the subgroups are small [think of the extreme case of having 100 samples, where category A has 96 samples and the others have 1 sample each] , it is likely that the bootstrap datasets will contain in fact only less categories and then this makes the estimation unfeasible
Sampling the residuals may help in this case, provided the dependent variable is numeric.
drawing weights from a dirichlet distribution (the bayesian bootstrap) is recommended in these cases, particularly when the dependent variable is a dummy variable with some regressors being categorical, which can easily end up in many bootstrap samples
Would it be correct, less technical and more bluntly to state that in some bootstrap, not every category is subsampled, e.g. X1 has five categories, but the bootstrap ends up with 1 (as in the extreme example)? Hence, in 2740 bootstraps not all categories are "selected".
@wil that's pretty much it. But it only happens when the relative frequency of a category is particularly small and when the model is applied to a different bootstrapped dataset.
If bootstrapping is implemented as a model with weights then it does not happen.