AIC and BIC are prediction criteria. Increased values imply the prediction will be better. Which is good if you want a prediction model. You don't give a research question so ??????.

I forgot to attach two papers you might find interesting. Best, David Booth

Sorry having some phone . Here's the other. Note the goals in each.

Thanks for the answer. But based on the R documentation: "When comparing models fitted by maximum likelihood to the same data, the smaller the AIC or BIC, the better the fit."

AIC for example is calculated as

2k - 2ln(L),

whereas k represents amount of parameters and L represents goodness of fit (in the form of your likelihood function).

Therefore, I agree with you Sepehr Mortaheb that a low AIC is what you strive for when estimating models using a maximum-likelihood approach. I think, that David Eugene Booth is right to point towards alternative approaches, but is wrong in his statement that better prediction will result in higher AICs, because the right-hand side of the equation indicates that with higher fit comes lesser AIC-values.

Concerning the parameter part of your question: as criteria punish for more parameters (see left-hand side of equation) as long as goodness of fit is not increased by a considerable amount, this might explain what happened in your models: Gender and age are predictors adding to the criteria without providing a substantial increase in fit as compared to a model without these predictors.

When aiming for the "best" model a model with lower AIC is thus better and you can discard unnecessary predictors, if this is your research question. Not including some predictors does not render the interpretation less transferable to the population because you do not alter the data. Different models only differ in their assumption about what factors have an effect on the data and gender and age do not qualify for such an explanation, in contrast to other factors.

Thanks to all for pointing out that my earlier answer was deficient in a number of ways in including but not limited to poor exposition. I still think I have a better solution than what has been proposed. First, if we look at the wikipedia entry for AIC we can summarize as follows: AIC favors models with fewer terms but it is well -known that AIC is a predominately predictive criterion. See our paper SRP- etc attached as well as our JDS paper also attached above. Using this criterion we would exclude gender and age to get the best predictive model. BUT we know that gender and age are reasonable explanatory models from the previous literature. It appears from reading the question that the author is interested in an explanatory model for he says that was why he included gender and age in the first place. If I were doing this and my goal was prediction where min AIC is important I would exclude age and gender as Sven Greving suggests. HOWEVER if my goal was explanation I care much lees about AIC and much more that age and gender have been good explanatory variables in other studies. Thus my solution is to look at the goal specified in the research question. . If the goal is the best predictive model I would not include gender and age. If the goal was explanatory I would include them because the goal and hence the solution is the best explanatory model. For more details, see the previously mentioned papers attached above. Further I offer a Mea Maximum Culpa for my poor exposition and earlier sloppy thinking. Best to all, David Booth

Thanks a lot Sven Greving and David Eugene Booth for the helpful comments. As Sven Greving mentioned, the amount of information that age and gender brings to my model is not really significant compared to the amount of complexity that it adds to the model. And that's why the AIC increases. I checked that by performing a likelihood ratio test between the two models and the test result was not significant. So, adding gender and age will not significantly change the model goodness-of-fit to my data. But David Eugene Booth is right about the study goal. Actually my goal is to develop an explanatory model for my dataset and therefore keeping these covariates would help explaining the results. But for prediction purposes, removing them would lead to better prediction accuracy. Thanks both of you for this fruitful discussion.

Best,

Sepehr

in what form is age being measured? I ask just in case you are using a large set of dummies; and there are more parsimonious ways of doing this that do not eat degrees of freedom.

Dear Kelvyn Jones ,

The age is in years old and is added to the model as a continuous variable. My model is something like this:

my_model = glmer (rt ~ fixed effects + age + gender + (1|subjects), data=my_data, family = Gamma(link="inverse"))

However I should mention that by adding age like that to the model the optimization problem for coefficient estimation cannot converge. Therefore, I scale age before adding it to the model:

my_data$age = scale(my_data$age)

Cannot get more parsimonious than linear age. More questions, sorry. Are the fixed effects - a set of dummies for the subjects. If so and Age is time invariant , it will be aliased out of the equation (or you get non convergence). But then do not understand why scaling (presumably grand mean centering) would help.

Actually in this experiment, subjects give a response to 4 different stimuli and so I just have a fixed effect with 4 factors.

rt ~ stimuli + age + gender + (1|subject)

and as you said age is invariant in this model. For the problem of non-convergence, different solutions were proposed such as changing the optimizer, restarting the optimization from the latest fit, etc. which none of them worked for me. But one of them was rescaling and centering the continuous parameters which worked fine in this case. The mean age in this study is 23 years old with a SD=2.9 years old. You think scaling and centering of age has a huge impact on the parameter estimation?

My mistake I thought you may have had panel fixed effects.

I would not think scaling should have a big effect on convergence except that without it, the intercept would represent a zero aged person and this would be far away from the data. There should not be a deleterious effect on the slope estimates.

I think I now see why there may be a lack of effect of age as with a mean of 23 and a SD of 2.9 (and presumably positively skewed), your subjects are quite homogeneous. Ceteris paribus, the smaller the SD of the predictor, the more imprecise the estimate, and it may also be that effect of age does not kick until you are older than your sample. But I am just guessing.