I know that in the Markov decision process (MDP), the probability of transition to a new state depends on the current state and chosen action of an agent. However, in my model, the new state of an agent also depends on the last previous action of itself and its neighbors. Can I solve the problem by a trick? The trick is to consider the last previous action of an agent and its neighbors to be a part of its current state space. I would appreciate it if you could let me know if there is any better solution.
my answer maybe not very professional.but...
as far as i remember i study a paper that he neighbors impact of giving the action selection(e.g greedy method).
Thanks dear Ebrahim,
I didn't understand your answer! Could you possibly explain more or send me the paper, please.
Therefore what you need is not a markov process. Try with markov random field or bayesian networks
Dear Negin,
May I know the algorithms that you will be using to solve the MDP? I am not very clear about your question, but you may try for SARSA Lambda algorithm that considers trace decay as parameter and considers eligibility traces as parts of the algorithm.
Thanks
Yes you can use your trick. In general, you can use a memory feature to maintain Markov condition as part of your agent state. However you will certainly fall down in large Markov decision process. The model will not permit you to consider more than a few neighbors to maintain an acceptable number of states and solve your model with traditional algorithms like value-iteration.
- Badges
- Science topic