I suggest you to contact Dr Yuting CHEN at Ecole Centrale de Paris whose recent PhD thesis tackles these algorithms

Basically filtration is suited to prospective analysis in real time as there is only a need for single step updates. BY contrast McMC requires refitting the model with an expanding dataset (over time) and possibly an expanding parameter set every new time step. Filtration is computationally more efficient for real -time as it only requires resampling and one step updating.

I think it depends on your model as well as on which inference task that you are interested in. If you are trying to solve the (on-line) filtering problem, then particle filters would be preferable for sure. Also for off-line inference tasks, smoothing and parameter learning, particle filters are well suited for dynamical models. If you haven't already, I would recommend having a look at particle MCMC,

http://onlinelibrary.wiley.com/doi/10.1111/j.1467-9868.2009.00736.x/abstract

http://jmlr.org/papers/v15/lindsten14a.html

which tries to get the best of both worlds.

If you have both continuous and discrete latent variables you can consider marginalising (i.e. Rao-Blackwellising) over the discrete states. This will reduce the Monte Carlo variance, but at the cost of increased computational complexity. Whether or not it is worth it depends on the model. This technique can be used both with particle filtering/PMCMC and with Gibbs sampling.

Dear Fredrik

Thanks for your helpful reply; I will try to read papers you mentioned, PMCMC sounds remarkable. Here I put some comment about my problem:

The initial DBN I had was very complex but I found out that it can be reduced to a slightly changed switching (or Markov jump) linear dynamical system (SLDS). There is a latent continues state X(t) which changes linearly but its dynamic model is switched by the value of a latent discrete state R(t). The observed variable Z(t) is a noisy version of X(t). The difference in my model with traditional SDLS is that here the transition probabilities of state R(t) is not only depended on its previous value R(t-1) but also on the previous value of continuous state X(t-1).

What I am interested in is the estimation of the sequence of state variables R(t) and X(t) t=1:T as well as learning the parameters of discrete state transition probability distribution P( R(t) | R(t-1), X(t-1) ) given observation sequence Z(t) t=1:T. I would like both tasks to be done on-line and simultaneously.

I know that it is possible to adopt particle filters for state estimation part. As in the following paper where Kalman filter is used for estimating the continues state and particle filter for discrete one.

http://ieeexplore.ieee.org/xpls/abs_all.jsp?arnumber=905890&tag=1

However, in such techniques the discrete state transition probability is assumed known while I need to learn it as well. This is why I started to consider MCMC.

Hi Vahid,

There are two things to consider here, first the fact that you want to solve the learning problem online, and second, how to best exploit the structure of the model.

i)

Online parameter learning is quite tricky in general - both MCMC and (hence) PMCMC are offline methods. If you are happy with a point estimate, you can try online EM. See for instance

http://arxiv.org/pdf/1312.0781.pdf

for a particle-based online EM algorithm which, in fact, is designed exactly for the type of model that you have (the discrete state is Rao-Blackwellised in this method). For Bayesian parameter learning, you could have a look at SMC^2

http://arxiv.org/abs/1101.1528

(which, however, you could argue is not really online) or particle learning

http://faculty.mccombs.utexas.edu/carlos.carvalho/LopesV9chapter.pdf

but then you should be aware of the degeneracy issue: http://arxiv.org/abs/1006.0554

ii)

Your model is an interesting extension of SLDS. For a standard SLDS, you have two options for Rao-Blackwellisation, either to marginalise the (conditionally) Gaussian state or to marginalise the discrete state. The former option is the "standard approach" for the Rao-Blackwellised particle filter, but unfortunately it is not applicable in your case due to the dependence on X in the transition probability for R. You can, however, still marginalise the discrete state (see the online EM paper mentioned above). You could also have a look at Section 3.4.2 in http://www.maths.bris.ac.uk/~manpw/apf_chapter.pdf for an alternative approach.

Thanks Fredrik, That was a great insight into the solution and good list of articles. I'll try to follow these guidelines and put here any updates.