I like to clear my confusion regarding MDP. It says only those process can be define by MDP that possess Markov Property. But, i think their are millions of other that don't posses that like in digital signal processing, control etc. If no, Is there any alternative for MDP?
I think there is perhaps a fundamental flaw in your implicit question. Those systems that enjoy the Markov property (perhaps in weak form?) can give us nice benefits that we can exploit in computation. Systems that do not enjoy this property cannot take advantage of the benefits. It would be unrealistic to expect that they would. If you want to use Markov based methods your system has to play by the rules. One hope though -- what may appear non-Markovian may in some instances be cleverly transformed. In my field this might include the mover / stayer model. (Divide the groups into two -- one of which has the Markov property.) I also like semi-markov processes where you can make assumptions about holding time distributions. Much mathematical modeling is in the art of designing your representation of the problem to fit a convenient/tractable form.
- Badges
- Science topic
- Similar topics
- Methodology
- Research Methodology
- Research Papers