The Likelihood ratio (LR) test has a null hypothesis that the two models, mixed effects and fixed logistic regression provide the same goodness of fit. Since P=0.1087, the null hypothesis can be accepted, and fixed effects logistic regression can be used as it is a simpler model.

Hey Adrian;

Thanks for your rapid response.

Shall I reply to the reviewers that a multivariate logistic regression provide the same goodness of fit so I´ll stick with the original?

Check that you specified the random effects properly first (maybe a time-varying coefficient)? Also, check with the different covariance structures. Compare them to the other model by running the simple model without random effects, than store it with "estimates store basemod", then run the lrtest command and compare with the new model. If none are sign, no random effects are needed.

Hi Lasse;

Its an one wave only cross-sectional analysis, so do no have time-varying coefficient.

On the other hand when I try a lrtest, storing a logistic regression and a multilevel logistic regression, stata gives me the following output:

"cannot compare a REML model with an ML model"

Thanks for your help

Thats not going to work. If you want to test this within the logistic regression framework, you can compare

logit outcome var var var

estimates store basemodel

to logit outcome var var var, vce(cluster "group")

lrtest basemodel

On a side note, its always a good idea to plot the predicted probabilities versus the standardized residuals

after logit:

predict pp, pr

predict sr, rstandard

scatter pp sr

.... and check for non-linearity

logit outcome var var var

linktest

or

mfp: logit outcome var var var

....and check that you found the optimal link:

glm outcome var var var, family(binomial) link(logit)

estat ic

glm outcome var var var, family(binomial) link(probit)

estat ic

glm outcome var var var, family(binomial) link(cll)

estat ic

glm outcome var var var, family(binomial) link(ll)

estat ic

lowest BIC is better (should perhaps be 5 lower, to go outside standard).

You can also check probabilties verus residuals, with the optimal link, in the GLM framework (help glm postestimation)

Back to your question, what you'r looking for is:

melogit outcome var var var

estimates store fixed

melogit outcome var var var || ID: group, cov(uns) or

Thanks Lasse;

I manage to read statistics with Stata and performed postestimation commands, but still have some doubts about the interpretation.

Output for Multilevel Logistic Regression

region | OR STD ER 95% IC

var(_cons)| .012162 .0134205 .0013987 .1057516

-----------------------------------------------------------------------------------------

LR test vs. logistic regression: chibar2(01) = 1.52 Prob>=chibar2 = 0.1087

* ACCORDING TO THIS: the null hypothesis can be accepted, and fixed effects logistic regression can be used as a simplier model.

Post-estimation

Is is still better to stick with a fixed logistic regression after this?

Random-effects covariance matrix for level region

_cons

_cons .0121618

Residual intraclass correlation

Level ICC Std. Err. [95% Conf. Interval]

region .0036831 .0040493 .000425 .031144

Thanks!

According to the results above, there is very little impact from random effects, and they should likely be dropped. Given that you have properly identified the covariance matrix, you should not need this random coefficient and can use fixed effects/standard log.reg. As mentioned above, make sure to try each possible cov(), f.ex. unstructured, identity, etc, and run estat ic after each: pick the one with the lower BIC. This is the _first_ step, before doing an LR test. If the _properly identified_ covariance matrix of a random effect is no better than fixed, according to LR test, I think that is quite strong justification for not including them. I guess, if a reviewer really wants you to adjust for invididual difference according to some group by random coffiecient, because it is established that you should do it (it probably isn't), you could include it as a "theoretical confounder". If you need more details, I will need more details on the problem, the covariates, the justification for the random effect, etc. Best of luck!

Thanks Lasse;

I´m still trying to interpretate both, random effect covariance and residual intraclass correlation. How could you explain the impact from random effects?

Random-effects covariance matrix for level region

_cons

_cons .0121618

Residual intraclass correlation

Level ICC Std. Err. [95% Conf. Interval]

region .0036831 .0040493 .000425 .031144

Thanks!

As stated above, you need to describe what you are trying to do (and why), in more detail. Why do you want to explain an insignificant finding?

Perhaps you can find some information here:

http://www.theanalysisfactor.com/the-intraclass-correlation-coefficient-in-mixed-models/