I shamelessly recommend Vandekerckhove, Matzke, & Wagenmakers (in press) and the references therein: http://www.ejwagenmakers.com/inpress/VandekerckhoveEtAlinpress.pdf

Edit: so Jeffreys weights are normalized marginal likelihoods. Take likelihood, integrate over prior, repeat for each model under consideration, divide each value by the sum of all.

DICw should be equally simply calculated as AICw. You first need to calculate the denominator for normalizing which simply is the sum of all exp(-0.5 * delta_DIC) [where delta_DIC is each DIC minus the smallest DIC]. Then each single delta_DIC value is entered into the same formula, exp(-0.5 * delta_DIC), and divided by the denominator.

I am not aware of any paper actually using this with DIC; but for AIC the formula is given in (http://www.ejwagenmakers.com/2004/aic.pdf):

Wagenmakers, E.-J., & Farrell, S. (2004). AIC model selection using Akaike weights. Psychonomic Bulletin & Review, 11(1), 192–196. doi:10.3758/BF03206482

-------------------

EDIT: As mentioned by Joachim in a different response, one should remove the 1/2 from the formulas.

Article AIC model selection using Akaike weights

If your models are nested you can use reversible jump MCMC to obtain posterior weights for each model.

Hi Gudeta

I really like this paper by Kenneth P. Burnham and David R. Anderson:

Multimodel Inference; Understanding AIC and BIC in Model Selection

SOCIOLOGICAL METHODS & RESEARCH, Vol. 33, No. 2, November 2004 261-304

DOI: 10.1177/0049124104268644

I think it’s great for sorting out BIC and AIC in model selection and model weighting for both frequentists and Bayesians. Might be worth checking out.

All the best,

Tom

Google

Bayes Information Criterion

(BIC appears in various softwares)

DIC model averaging has been discussed several times in the literature, but there is no theoretical formulation of how to do this or why it should work. DIC is a corollary to AIC, but is problematic in interpretation for some models- specifically hierarchical models (models written out conditionally with no singular integrated likelihood). The classic Bayesian model Averaging work is written by Jennifer Hoeting at Colorado State University. See Hoeting, Jennifer A., et al. "Bayesian model averaging: a tutorial." Statistical science (1999): 382-401.

Sincerely, Brian

The following paper illustrated the DIC measure and the ambiguous relation between DIC and AIC:

Spiegelhalter, D.J., Best, N.G., Carlin, B.P., and Vander Linde, A. (2003), “Bayesian measures of model complexity and fit (with discussion and rejoinder),” Journal of the Royal Statistical Society, Series B, 64, 583–639.

Different criteria are based on different grounds. DIC is based on predictive power of the model. Hence, if you use DIC, then you are comparing models according how well do they predict. BIC is based on the assumptions that asymptotically, the true model is from exponential family, so that on small samples I would not suggest using this criterion. There are many more other criteria, each with it's own assumptions. And obtaining one criteria from another is just unreasonable due to the same reasons of different assumptions.

My suggestion would be just to calculate directly each criteria you want to use, it's not difficult, especially if you are runnign Bayesian analysis with some MCMC sampler.

Cheers.

To compute DIC weights, use the same formula as AIC weights but without the 1/2 in the exponents. That is there to undo the multiplication by 2 in the AIC computation, which the DIC doesn't have.

An even nicer Bayesian option is Jeffreys weights, which takes the Bayesian evidence (likelihood integrated over the prior space) for each model and normalizes by the sum of evidences over all models. It is something like a multi-alternative extension of the Bayes factor.

DIC is not completely Bayesian. See "Understanding predictive information criteria for Bayesian models" http://www.stat.columbia.edu/~gelman/research/published/waic_understand3.pdf

There is no theoretical justification for using DIC (or WAIC) for model averaging. You should use posterior probabilities.

I would like to thank all for the comments

Joachim, I find what you called "Jeffreys weights" very appealing, if I have understood correctly it is likelihood integrated over the prior space. I wish I could get published literature on that. Do you happent to know any paper on that?

I shamelessly recommend Vandekerckhove, Matzke, & Wagenmakers (in press) and the references therein: http://www.ejwagenmakers.com/inpress/VandekerckhoveEtAlinpress.pdf

Edit: so Jeffreys weights are normalized marginal likelihoods. Take likelihood, integrate over prior, repeat for each model under consideration, divide each value by the sum of all.

Many thanks Thomas,

I have tried maximum likelihood estimation (which gave AIC and BIC), but I am more interest in Bayesian with MCMC using Gibbs sampling. I have the DIC and posterior estimates with their 95% credible intervals. I am comparing 4 non-nested models, and the difference between DIC values is in the range of 8-20. For sure I cannot say, the model with the lowest DIC is the best. So I wanted to try weighting just like we do AICw. I guess, I can now calculate these weights just as Henrik suggested but "without 1/2 in the exponents" as Joachim says.

In case you have very close values of the criterion for your models, I would suggest for graphical "interogation". Replicate the data from your model and compare them graphically to the data you are fitting to.

The ins and outs of Akaike weights is explained in a clear way in Kendall's Advanced Theory of Statistics, vol. 2B Bayesian Inference. In section 7.50 and further. Irefer to that excellent publication.

The counterpart of AIC, BIC or DIC is the evidence p(D|Ci) in Bayesian, where D is the data and C1, C2,... are the model classes. (In fact, it will be -1/2 ln (evidence) that can be compared to AIC or BIC.

Thanks Yuen for that. I will be happy if you could point me towards a paper or book for further reference.

I am glad to do this. You may either read my paper Beck and Yuen (2004) (this can be downloaded in Researchgate) or my book "Bayesian methods for structural dynamics and civil engineering" published with John Wiley and Sons. Feel free to let me know if you have further questions.

Call for Papers:

"12th International Probabilistic Workshop" held in Weimar Germany, on November 4th and 5th 2014"

https://www.researchgate.net/post/Call_for_Papers_12th_International_Probabilistic_Workshop_held_in_Weimar_Germany_on_November_4th_and_5th_2014

Interesting discussion

but the reason why the formula for DIC weights is the same of AIC ones but "without the 1/2 in the exponents" still puzzles me. Can someone provide a somehow more formal lik/explaination?

thx

The formula for DIC is a function of negative log likelihood, and AIC has -2 log likelihood. You need to remove that -2 to get to the right likelihood scale. That's all there is to it.

well, I think that the -2 is there also for the DIC, isn't it?

DICk=-2 E[log(Lk)] + 2 pD 

where pD=Var(D)/2 following Gelman et al 2004

am I wrong?

No, that doesn't seem wrong.  It might be that there is a likelihood in the pD.  It's been a while since I looked into this, let me get back to you in a bit.

Bayesian Model Comparison possesses an in-built Occam's razor. I can refer to Jaynes' Theory of probability (2003) or Gregory's book on Bayesian Logical Data Analysis. The Akaike stuff is, hence, not needed as a separate device to judge two models on their relative simplicity. I have always found it strange that some Bayesians invoke  it. On top of that, it assumes Gaussianity (lots of data).