Does it make sense to include a cross-level interaction term in a multilevel model without specifying a random slope for the Level-1 variable?
I am well aware that a cross-level interaction effect between variables X (level 1) and Z (level 2) can be tested, even if X has no significant random slope (see Snijders & Bosker, 1999, p. 96). However, there is clearly a difference between an insignificant random slope and not including a random slope term at all in a cross-level interaction model.
The reason I'm asking:
I have vignette data at level 1 nested within individuals at level 2. I want to test a cross-level interaction between "context" (a vignette-level variable) and "gender" (an individual-level variable). I am not interested in testing whether the effect of the vignette-level variable varies randomly across individuals (random slope model). I am only interested in testing whether the effect of the vignette-level variable varies between boys and girls (model with cross-level interaction effect). However, I'm wondering about the extent to which the effect and the significance of the cross-level interaction term in the model depends on the inclusion of the random slope term.
Many thanks in advance,
Arne
Put simply no. A cross-level interaction is just an interaction where one of the predictors is restricted in its variability to units at level 2. If the model makes sense then go ahead.
It is however not uncommon for a random slope to be an equally good explanation (in terms of model fit) to a model with the cross-level interaction. In that case the plausibility of the model is what matters most. See link to more detailed answer by Jake Westfall.
http://stats.stackexchange.com/questions/59850/can-i-probe-cross-level-interactions-without-random-slope-in-a-hierarchical-line
Thank you Thom, but could you please further clarify your answer. First you say "put simply no" (so, it doesn't make sense to include the cross-level interaction without the random slope term ?), later you say "if the model makes sense, go ahead".... I'm confused.
I'm familiar with the discussion at Stats.stackexchange, but I'm not sure whether it really answers my question. Jake Westfall writes: "In the cross-level interaction model itself, it is entirely possible for there to be no variation in the level-1 slopes". I agree with him, but his statement is one about the variation in the random slopes, not about the inclusion of the random slope term in the model. To what extent is it necessary to include the random slope term (mathematically speaking) ? To what extent does the cross-level interaction effect (in terms of its size and significance) depends on the inclusion of the random slope term in the model ?
The micro and macro specification in the manner of Raudenbush and Bryk makes it clear that the macro variable involved in the cross-level interaction is trying to account for unexplained differences in the random 'slopes' (or random differentials in your case of gender as the level 1 predictor). Moreover there could be unexplained differentials for the higher level unit that remain after the inclusion of a macro variable and correctly estimated standard errors need to take this into account. For both these reasons I would model the random differentials at level 2 associated with the included level 1 predictor. While I do not agree going maximal in terms of random coefficients with each and every predictor (with concomitant problems of over specification and non-convergence) I think that is important that you do so here - you may be interested in this submitted paper
https://www.researchgate.net/publication/299604336_Fixed_and_Random_effects_making_an_informed_choice
Article Fixed and Random effects models: making an informed choice
Thank you for your answer Kelvyn.
MLwiN allowed me to include a cross-level interaction term in my model without having to include a random slope term. Moreover, I get perfectly reasonable results in line with my theoretical predictions. The cross-level interaction term indicates that the slope of the level-1 variable varies across gender (which is the level 2 variable in my analysis, as responses are nested within individuals). If I understand you correctly, the standard errors of my cross-level interaction effect are not correctly estimated unless I also include the random slope.
Yes on both counts
1) Mlwin would do that - it does not know what is the conceptual status of the variable.
2) the SEs could be affected although some elements of the dependency will be picked up through the 'random intercepts'
It would help if you would explain why you are reluctant to move to a quadratic or linear variance function at level 2 involving the boy/girl dummy- what are the perceived 'costs' to you of doing this?
Thank you for the clarification Kelvyn.
The clustering of the measures within individuals is not of substantive interest to me. My first idea was to simply use a GEE approach, but then I remembered that Mlwin can easily be used to take the clustering into account (random intercept model) AND calculate population-averaged values via the simulation of predicted probabilities (my outcome is ordinal). I assumed the random intercepts would be sufficient to pick up the dependency and keep my model as simple as possible. I'm not interested in estimating how much variation there is in the slopes, only whether they differ between boys and girls. However, now I understand that if I do not model the variance correctly, the SE could be affected....
In addition to the answers already given by Kelvyn Jones and others, I refer interested readers to a recently published article on this topic in European Sociological Review:
Jan Paul Heisig, Merlin Schaeffer, Why You Should Always Include a Random Slope for the Lower-Level Variable Involved in a Cross-Level Interaction, European Sociological Review, Volume 35, Issue 2, April 2019, Pages 258–279, https://doi-org.kuleuven.ezproxy.kuleuven.be/10.1093/esr/jcy053
I agree with Arny. It is not compulsory to have random slopes with cross interactions. See my paper with citation
Jayawardena, N. and Sooriyarachchi, M., 2014. A multilevel Bayesian analysis of university entrance eligibility for selected districts in Sri Lanka: methods and application to educational data. Journal of the National Science Foundation of Sri Lanka, 42(1), pp.23–36. DOI: http://doi.org/10.4038/jnsfsr.v42i1.6676
If you have any questions about this paper I will be willing to help.
Thank you Arne for starting a discussion and getting answers from Kelvin. It is now clear for me that some publications got it wrong, including one of me.
- Badges
- Science topic
- Similar topics
- Mathematics
- Statistics
- Sampling Design
- Sample Size