I have two candidate models, Model 1: y~x1+x2; Model 2: y~x2. I am using the brms package in R and have used the default non-informative priors (There is no existing literature to draw my priors from, ergo my choice of N-I priors). The distribution is bernoulli. The model is causal, rather than predictive. Given these conditions, I would like to check which model is more useful and then cite a statistic to support my choice. In frequentist glm, this was AIC. Is there something similar in Bayesian glm? (I was thinking WAIC). I have used AUC and Model 2 gives me a value of 0.68 vs 0.61 for Model 1.
I am very new to the Bayesian side of things and a biologist. I would appreciate any suggestion or recommendation of papers/books to nudge me in the right direction for my question
The bayesian value that is equivalent to AIC is the DIC ( Deviance Information Criterion). This is the equivalent value in Bayesian approaches which you can use for model comparison.
Thanks
There are different measures that you can use, for instance, the Bayesian information criteria (BIC) used jointly with the Deviance information criterion( DIC). An improved discussion and compression can be seen in
Article Assessing the practical differences between model selection ...
Rohan Sarkar , the Schwarz information criterion is in fact the Bayesian-closely related cousin of AIC. You want to explore also the theory about Bayes factors.
Abdisalam Hassan Muse : Thank you for your answer
Pedro Luiz Ramos : Thank you for your answer and article suggestion. It seems to me that since my priors are uninformative, the DIC might be the way to go (even WAIC will do), as it more liberal and doesn't penalize my choice of broad priors (If my understanding of the article is correct). Would "loo" be one way to check the same thing? The article doesn't make any mention of that
Rolando Gonzales Martinez : Thank you. I read that Bayes factor depends heavily on the choice of priors and shouldn't be relied upon too much when using uninformative priors (I am still learning so my knowledge might be incomplete at this point. So feel free to correct me). The bayes factor in Model 2 was 4700+ (I am not sure if this is okay)
Rohan Sarkar , yes, this is the case because even in large samples the prior appears in the marginal likelihood that is used to calculate the Bayes factor; there are exceptions to this situation (when the parameters are observed-orthogonal) and solutions (sensitivity analysis to see the effect of priors or using the Schwarz criterion as an approximation to the Bayes factor - the last one is my favorite one and the one I personally use as it works when n -> infty+). Please see:
https://www.jstor.org/stable/2348679
About your Bayes factor, it is a value quite surprisingly high providing decisive evidence, which I never encounter myself in real life model applications, it has sense if the density of the parameter associated with x1 is highly concentrated around zero, but maybe you want to double check your calculations ;)
(In my opinion it has sense to prefer Model 2 because the AUC of Model 2 is higher than that of Model 1, too)
- Badges
- Science topic
- Similar topics
- Geoscience
- Asia
- India