Hi,
I'm using the lme4 package in R to run fairly simple linear mixed effects models. I'm unsure about how to report confidence intervals (CIs) for fixed effects estimates. I know that CIs can be extracted with confint.merMod(model) and that there are 3 methods available for this: profile, Wald, and boot. I have read in Bates et al (2015) (Article Fitting Linear Mixed-Effects Models Using lme4
) that the three methods produce similar results, but I get an error message with the profile method for some models, but not others. I am yet to find a published article that explicitly states which method was used to compute CIs, so I'm assuming most people go with the default 'profile' method, but ideally I would like to know what each method implies. I can't find anything online, so it would be great to hear from anyone who has run such models and reported CIs (which method did you use and why? did you specify the method?). Reference to relevant resources would also be welcome.Thanks, any help appreciated!
Chris
Bates has discussed this (and the related p value question) on listservs and elsewhere for a few years. A brief summary is at http://127.0.0.1:14171/library/lme4/html/pvalues.html
The following is also a useful vignette
https://cran.r-project.org/web/packages/merTools/vignettes/Using_predictInterval.html
I'm interested what other sources people provide to this question.
I don't have an answer for your question, but joining this newsletter with Bates and others who run this package has been useful for me.
"To subscribe or unsubscribe via the World Wide Web, visit https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models"
This widens the answer in relation to Daniel Wright 's request
MLwiN ( which can be accessed from R via http://www.bristol.ac.uk/cmm/software/r2mlwin/ ) has MCMC estimation as well as likelihood capability. This has a number of advantages in that the posterior distribution provides
1) what your really want - that is credible intervals and not confidence intervals.
" The latter are a frequentist concept and describe the anticipated performance of the estimate across repeated samples. If you had 100 repeated samples from the population and you constructed 95% confidence intervals for each sample, you would expect 95 of the intervals to include the population parameter. The probability statement therefore applies to the data not to the parameter. This is also the case for a null-hypothesis significance test. It is the proportion of repeated samples that would give a test statistic equal to or greater than that of the data. In contrast, the Bayesian credible interval is more natural and applies to the parameter and not the data, giving a 95% probability that the parameters falls between the lower and upper values " "The Bayesian approach, has a very different perceptive on inference and does not concern itself with repeated samples from hypothetical super-populations. Rather, it asks the question what is the degree of support for a parameter, combining the researcher’s beliefs about the parameter and the evidence from the data? The answer to this question is given by the posterior distribution and its associated credible intervals." Bayesian estimation is sometimes known as full uncertainty analysis as the uncertainty in all parameters - fixed and random - are taken into account simultaneously.
2) in practice , you do not usually get much difference for the fixed effects estimates (unless sample size is small or you impose a tight prior) but you can get potentially asymmetric credible intervals; this is particularly important for the estimation of the variances of the random effects which because the estimated variance cannot go negative is likely to have a positively skewed distribution; my understanding is the lme4 does not provide anything because the normal approximation is often likely to be poor;
3) the credible intervals are also provided for the covariances if random slopes are used, indeed you can also get the intervals for correlations between random intercepts and slopes without involving tricky normalizing assumptions or transformations;
4) Apostasy! You can even get (for the fixed effects) Bayesian p values; representing posterior tail probabilities
It is very easy to use and very fast fast (for MCMC) to estimate, see Chapter 10
in Book Developing multilevel models for analysing contextuality, he...
@Chris Dixon; Sorry for hijacking you question!
@Chris Dixon , , Perhaps you haven't read the R Package information about the use of lmer4, warning on getting error message (perhabs from misspecification, sample in use for the likelihood models) and that the recent versions of lme4 (e.g. 1.1-6) give false convergence warnings.( i.e., you are seeing a false-positive warning). New or future versions (1.1-7 and up will correct for this limitations as hoped by the developers, and R team. {this is meant to throw some light, not really an authority views}
#REPORTING: Trustworthy CIs are much preferred to p-values. In reporting the confidence limits, you need to make statements on the method (the choice, both effect or just the fixed effects) and also on the uncertainty: For instance, "The effect apprears to be is strong, but the CIs are wide. We cannot exclude that it is much weaker [or stronger] and the chance of possible false values (true values not falling within the CI is.... "
or ,as appropriate, "The effect is practically zero and we can say so with good certainty"
On the #METHODS:
"profile": this is by computing a likelihood profile and finding the appropriate cutoffs based on the likelihood ratio test. I would prefer this.
"Wald": approximating the confidence intervals (of fixed-effect parameters only; all variance-covariance parameters CIs will be returned as NA) based on the estimated local curvature of the likelihood surface;
"boot": performing parametric bootstrapping with confidence intervals computed from the bootstrap distribution according to boot.type (see bootMer, boot.ci). A problem with the new version include report of non convergence. The method is at least six times slower than other approaches.
So just found out about this package called report and you run the model, then report(model) and it gives you exactly what you need to report. You have to be able to interpret the results but I thought extremely useful for writing and not wasting my time with 3000 different packages to calculate all the stats needed. It works for other tests too (ANOVA, etc).
- Badges
- Science topic
- Similar topics
- Correction