In AlphaGo Zero algorithm, the neural network returns a pair (p, v) consiting of a vector of move probabilities p and a scalar v indicating winning probability. The v is used for node evaluation during MCTS rollout. But is p used anywhere during MCTS procedure?
I understand that the network is trained to approximate search probabilities π computed by MCTS with p, but I am not sure whether p is used during MCTS itself.
After in-depth reading of original paper, I belive I have found the answer. The p vector is used as an initial policy during every node expansion. The MCTS then provides improved vector π - therefore, MCTS can be seen as a policy improvement operator.
I suggest you can understand the core concept of your query here: https://discovery.ucl.ac.uk/id/eprint/10045895/1/agz_unformatted_nature.pdf
which is straightforward. More institutively, it is here too : https://medium.com/@jonathan_hui/alphago-zero-a-game-changer-14ef6e45eba5
- Badges
- Science topic