Do you have the raw data? If so it seems that using a logistic regression to model the probability of a hit would be the appropriate approach (or maybe a probit model - especially if you are interested in signal detection) using glmer. A multilevel beta regression might also be possible (but that assumes no proportions are zero or 1).

Do you have the raw data? If so it seems that using a logistic regression to model the probability of a hit would be the appropriate approach (or maybe a probit model - especially if you are interested in signal detection) using glmer. A multilevel beta regression might also be possible (but that assumes no proportions are zero or 1).

I agree with Thom Baguley that a logistic regression where the outcome is 'hit' or 'miss' would be appropriate. Try using the following:

install.packages("glmm")

library(glmm)

model

Hello María Quirós-Godoy , In addition to the above response, 'function' can be used where you can use IF and ELSE argument to set it as per your wish the algorithm will execute.

Look: https://www.datamentor.io/r-programming/if-else-statement/

Good Luck!

Thom Baguley and Blaine Tomkins thank you for your contributions. At first, I was working with the proportion of hits as data outcome. I tried with glmer and "family=binomial (logit)" function but I was not sure about the results because, as you say, I think it would be better to use "hit/1" or "miss/0" as outcome. Luckily I have access to raw data, so I'll try with it.

Blaine Tomkins I'll try your idea about using random slopes with age. This is my first time with multilevel and I'm learning about it day by day. That kind of comment is very useful for me. In the case I use random slopes, I´d have 2 random effects, one for random intercepts (participants) and one for random slopes (Age), isn´t it? How can I plot random slopes? I'm using this function to plot random intercepts:

plot_model(M11_Int6, type = "re", sort.est = "sort.all", grid = F)

I'm interested in the interaction between Age and the other variables to study how they develop, that's why I'm using Age as independent variable.

Suraj Bhagat thank you for your contribution too, I'll take into account if I have problems with glmer and the binomial distribution.

I'm still open to new ideas about this topic.

It sounds like your outcome variable is a proportion between 0 and 1. In this case, as Thom Baguley suggests, a beta regression may be appropriate. In R, to include random effects with beta regression, you might use a glmm, for example with the package glmmTMB (though there are others; but, note for example that the glmm package does not include beta regression as an option). But as was noted, typically beta regressions do not accept values that are precisely 0 or 1. Sometimes people will make a slight adjustment to 0 or 1 values to make them accepted by the beta regression (but I'm not sure this is always a good idea).

A different approach might be to just stick with the lmer model you have and deal with the fact that some predictions are outside the logical bounds of your outcome variable. There may be some advantage to use the relatively common lme4 package, simply because readers and reviewers will recognize it, and because it is supported by other packages e.g. lmerTest and emmeans. (It appears that there is some support for emmeans in glmTMB, but I'm not real familiar with what's available or supported.) It may be worthwhile to run the model as a beta regression in glmmTMB and compare it to the lmer model. It may not make much a difference except you having to explain that some predicted values are greater than 1, which still may tell you what you want to know.

María Quirós-Godoy in case you are open to new ideas, then it will be several possible options to select.

Good Luck!

You can use logistic regression with proportions as long as you have the numbers in each cluster, you don't need the individual 0s and 1s unless you have predicters at that level.

María Quirós-Godoy You can disregard my last suggestion. I (lazily) neglected to re-check your model, where in fact you do have a random slope for age itself.

At the risk of overfitting, you might also include a random intercept for target (1|Target) in the model. (Granted, I don't know what these targets consist of, and if there's reason to believe specific targets would influence accuracy).

Mixed-effects modeling allows for a wide range of complex hierarchical models. For instance, suppose there is a large degree of individual differences regarding the influence of set size. You could construct a nested model with 'set size' nested in 'subjects' as such: (1 + SETSIZE|Subject), and compare this to a model without this assumption. The options are numerous, though that is also a risk with hierarchical modeling (and all modeling, really). It's easy to get carried away to the point where your model becomes so complex that it loses it's usefulness for predicting new data sets.

@Maria

Just to add how the logit (Loge [P/(1-P)]), where P is a proportion, works in simple terms. A proportion of zero is a logit of minus infinity, a proportion of 1 is plus infinity. When you model the underlying log odd of 'Yes' you cannot be outside the bounds of -/+ infinity so when converted back the predicted proportions must lie between 0 and 1 ; this is in effect, the constraint you are looking for.

I use them term 'underlying' logit as it is not the observed proportion that is transformed (the empirical logit) but the predicted or estimated value; the so-called systematic part of the generalized linear model. It does this as part of an iterative scheme that maximizes the likelihood.

https://statisticaloddsandends.wordpress.com/2019/10/31/understanding-the-components-of-a-generalized-linear-model-glm/

I would only add that the above advice looks good. best wishes to all, D. Booth

Thank you for all of your responses. Finally, I followed the suggestion of Thom Baguley and used glmer with the raw data. It solved my problem (thank you Thom!) But now I have another one :(

My model includes some interactions, and I want to make pairwise comparisons of the predicted probabilities between conditions but I'm only able to compare the odds, but not probabilities. This is my code for pairwise comparisons:

Mean_V1_V2

pairs is the R function for a scatter plot matrix.

Since the predicted probabilities are just a function of the odds ratio, I'm not sure what the problem is, but it is probably worth starting a new question saying what you want and providing output.

First, I'd recommend using adjust="hochberg" or "hommel" as the Bonferroni correction is too conservative in this case.

Second, you need to think about what you are asking. The predicted probability is a function of the other predictors in the model so there may not be a sensible pairwise comparison on the probability scale unless you hold the other variables constant. If you just have grouping variables/factors it is possible.

In the former case I plot the predictions as a function of the other predictors.

emmip(gm1, ~ period, type='response')

The simplest solution is to run the tests on the log odds or odds scale and interpret the plot or differences in the probabilities in conjunction with those tests. Getting the difference in probabilities is simple arithmetic once you transform the group predictions (using emmeans to get those or a model with all other effects centred). I think you could get the CIs on the probability scale but it is fiddly and may not be necessary. I'd usually be happy to work with a plot of the predicted probabilities for interpretation.

For a simple group 95% CI you can run the model with a cell means parameterisation and use confint():

gm0

Another fantastic response from Thom. I would just like to add in case you want differences of proportions instead, you could run: pairs(regrid(model)).

Thank you Daniel Wright and Thom Baguley . In this case, all my variables are categorical. This problem has arisen when I tried to interpret the results related to the graphs I'm attaching. In the odds plot, for example, the difference between group 0 and 1 was no significant, but when I transform to probabilities and attending to the CIs, there were differences, but I was thinking of using p values too. My research question is if there are differences in accuracy between groups.

I have 34,483 observations, is this enough to use CIs as a good method to interpret the differences?

About the method to adjust, I was reading and trying different methods, indeed Hommel was one that I read about, do you know some articles to report that that method is more adequate? About that, In one of my analyzes, I used emmeans and lsmean for post hoc comparisons, and while with the first I did not obtain differences, I did with the second one, but I am not sure that using least squared means without correction like Bonferroni it can be correct to use them.

Sorry for all these doubs, but I don't have no one close to me to ask and I'm triyng to make everything by myself.

I opened a new question with this topic but I didn't obtain any response, that's why I wrote here again looking for your help (and I really appreciate it!)