I have several 4-state Markovian processes. Is there any meaningful way to classify those processes based one their stochastic matrices? Perhaps with respect to stability? How about processes that have different numbers of states?
I think that the classification based on the eigenvalues of the stochastic matrix is possible. Specifically, the second largest eigenvalue is sometimes used for estimating of the mixing time of the Markov chain.
In general, the evolution of the distribution or the transition probabilities with time n is fully described by the eigenvalues. Actually, the transition probabilities are the linear combination of lambda_j^n * p_j(n), where lambda_j is the eigenvalue and p_j(n) is some polinomial of degree (s_j-1) where s_j is the multiplicity of lambda_j in the characteristic equation. So, the eigenvalues determine the rates of convergence to the limiting regime (if any).
- You can refer the book " Operations Research - An Introduction " 8th Edition, ISBN 0131889230. Pearson Education, 2007 by Hamdy A. Taha. This book will help you.
The first step is to determine the nature of your stochastic processes about the time of transactions between states (discrete or continued). There are different types of Markovian processes and if the time is discrete you can use Markov Chain (MC) apparatus. To define a MC you must determine: 1. Set of states (discrete and final); 2. Matrix for probability transactions (homogenous matrix); 3. Vector of initial probabilities (for starting the modeled process). For each process should be designed separate MC model (N+1 equations for N states) and it could be analytically processed for determining final probabilities for all states (stationary regime).
- Badges
- Science topic
- Similar topics
- Biological Science
- Anatomy
- Tissue