This Fall, I am teaching my master students an introductory course on Markov chains. I am looking for an easy/clear method to explain the theoretical basis of simulating a Markov chain; i.e.; theoretically how to mathematically explain simulating a Markov chain.
I know one can easily simulate a Markov chain using Mathematica or the R package "markovchain", but I need to do it manually by drawing random numbers from unif[0,1]. Thanks so much for your answer.
Mohamed Riffi
Markov Chain (MC) is a stochastic apparatus and definition of MC-model is a system of N+1 equations for N probabilities. To determine the final probabilities this system could be solved manually or by universal program system.
Markov Chain (MC) is a stochastic apparatus and definition of MC-model is a system of N+1 equations for N probabilities. To determine the final probabilities this system could be solved manually or by universal program system.
Markov Chain (MC) is a stochastic apparatus and definition of MC-model is a system of N+1 equations for N probabilities. To determine the final probabilities this system could be solved manually or by universal program system.
For discrete-time Markov chains, it is quite simple. Suppose M is the transition matrix of the process, such that p(t+1) = Mp, and hence each column of M represents the conditional mass function for X(t+1), given X(t).
At each time, observe the current state of the chain, and then take the cumulative sum of the X(t)'th column of M, to obtain the conditional distribution of X(t+1), given X(t). Now, take a uniform random variable on the unit interval, say U. If U falls between 0 and the first element of the cumulative sum, assign X(t+1) = 1. If U falls between the first element of the cumulative sum and the second, assign X(t+1) = 2. In general, if U falls between the k-1'th element of the cumulative sum and the k'th, assign X(t+1) = k.
You can see that by construction, the distribution of the induced process {X(t)} is exactly the distribution of the desired chain.
Something similar (but conceptually somewhat more involved) works for continuous-time Markov chains. If you need more information on that, look up the Gillespie algorithm. It's basically a stochastic variant of the Euler method, adapted for stochastic processes.
Dear Radi Petrov Romansky,
Thanks so much for your answer. Theoretically, I know how to find the stationary probability distribution of a finite discrete-time MC. My question about how to simulate such a MC.
Best Regards'
Mohamed I. Riffi
Dear Nicholas J. Watkins,
That's exactly what I am looking for. Thanks so much for your answer.
Best Regards'
Mohamed I. Riffi
Dear Benjamin Peter,
I already know how to do that using Wolfram Mathematica, but I was asking about manually (the algorithm) simulating a finite state discrete-time Markov chain.
Thanks for your answer.
Mohamed I. Riffi
Is there any notes that i refer on how to simulate Markov chain using R. Tq
- Badges
- Science topic
- Similar topics
- Computer Communications (Networks)
- Simulators