For example, what is the difference between "On-Policy Temporal-Difference Learning" and "Off-policy Temporal-Difference Learning"
I think you should read in the link I herein attached for you. Their discussions were useful and direct to your questions.
https://groups.google.com/forum/#!topic/rl-list/4Efnr0gXhAU
hi, I recommend you this website:
https://www.cse.unsw.edu.au/~cs9417ml/RL1/tdlearning.html
On-Policy and Off-Policy Learning
On-Policy Temporal Difference methods learn the value of the policy that is used to make decisions. The value functions are updated using results from executing actions determined by some policy. These policies are usually "soft" and non-deterministic. The meaning of "soft" in this sense, is that it ensures there is always an element of exploration to the policy. The policy is not so strict that it always chooses the action that gives the most reward. Three common policies are used, e-soft, e-greedy and softmax. These are explained in the section below.
Off-Policy methods can learn different policies for behaviour and estimation. Again, the behaviour policy is usually "soft" so there is sufficient exploration going on. Off-policy algorithms can update the estimated value functions using hypothetical actions, those which have not actually been tried. This is in contrast to on-policy methods which update value functions based strictly on experience. What this means is off-policy algorithms can separate exploration from control, and on-policy algorithms cannot. In other words, an agent trained using an off-policy method may end up learning tactics that it did not necessarily exhibit during the learning phase.
- Badges
- Science topic