For my Matlab code, as soon as the number of random variables becomes 3, acceptance rate of MCMC using metropolis-hasting algorithm drops to less than 1%.
It sounds like your MCMC is suffering from the curse of dimensionality - the more parameters you have the smaller the chance of jumping to a new location in multivariate space that has any reasonable probability. In such situations it is often useful to tune your MCMC (either manually or automatically) before running the full thing. As a rough sketch of the method; if the acceptance ratio of the last n (maybe a couple of hundred?) iterations is less than 20% then decrease the size of your proposal density. If it is larger than 20% then increase the size of your proposal (for some reason 20% is widely taken to be an 'acceptable' level of mixing). You'll want to do this separately in each dimension, as your parameters may have different levels of dispersal.
Other ideas: if your parameters are highly correlated then you may be moving very inefficiently around the posterior - if there is a diagonal ridge in the posterior space then jumps in either x or y independently may be unlikely. One idea would be to fit a multivariate normal (with correlation terms) to your burn-in, and then make this your new proposal.
Hello, Sharvil! I'm not sure about your problem, but I think that you should read about Successive Projections Algorithm (SPA). It is a very helpful tool for variable selection in multivariate calibration problems and perhaps it may be useful for your problem too.
Best wishes and good luck with your research!
Hi, Sharvil!
Try "emcee" - python package implementing affine-invariant ensemble sampling. No turning is required and it can easily sample from very weird (correlated a-la banana shaped) and high dimensional distributions. All you need is to code your density (from which to sample - your posterior, i suppose) as python function and pass it to emcee constructor. It is well tested and used by many scientists in refered astrophysical publications.
Ilya
P.S. If you need MATLAB implementation of affine invariant ensemble sampling - you can code it yourself. It is quite easy algorythm.
I just wrote my own version of the affine invariant algorithm for matlab (similar to emcee). perhaps you might find it useful:
http://www.mathworks.com/matlabcentral/fileexchange/49537
You need to choose a different proposal density. If that does not help you may want to use some kind of adaptive MCMC algorithms.
- Badges
- Science topic