For instance you are investigating gaps in 2 independent variables and you expect it to close up soon. How will Markov Chain n-step transition matrix be able to interpret when these gaps are likely to closeup?
The simplest way is to modify the trasision probability matrix by cutting off exists from the states accepted as terminating the process. For discrete state space S , if A \subset S is the subset of states terminating the analysis, then for every j \in A one should replace all Pj,k by 0 for k not equal j, and ((consequently) Pj,j=1. Then for the modified process Xt the probabilty
Pr{ Xt \in A} = \sum{j\in A} Pr{ Xt = j}
is an increasing function of discrete time t, and equals the probability
Pr{ TIME_TO_REACH_A \le t }.
Final formula:
Pr{ TIME_TO_REACH_A = t} = Pr{ Xt \in A} - Pr{ Xt-1 \in A} for t=1,2,3,...
REMARK. if the initial state is in A, then all Pr{ Xt \in A} equal 1, and if the initial state is outside A, then Pr{ X0 \in A}=0.
REMARK. If lim Pr{ Xt \in A}
@Joachim, Thank you for the suggestion. Do you mean I should treat A as an absorbing matrix, a subset in S?
YES! More precisely, for your purposes you are replacing the original transition probability matrix say P by the one Q with changed exits from the chosen set A subset S . This MAKES the set A absorbing.
Let me provide a very-very simple example: S = {0,1} and
P= [ 1/2, 1/2; 1/2, 1/2] .
Then the first entrance into 1 from 0 is geometrical on 1,2,3... with ratio 1/2, i.e. Pr{ T \le t} = 1 - \sum{n=t+1}infty 1/2n = 1 - 1/2t , t=0,1,2,.... For calculating this distribution according to my suggestion, you build a NEW matrix:
Q = [1/2, 1/2; 0, 1], Then the t-th power of Q equals
Qt = [ 1/2t , 1 - 1/2t ; 0 , 1 ] t=0,1,2,....
Thus Pr{Xt = 1| X0 = 0} = 1- 1/2t which agrees with the above easily expected result for Pr{ T \le t}.
Regards, Joachim
- Badges
- Science topic
- Similar topics
- Mathematics
- Algebra
- Matrix