In the field of reinforcement learning based optimal control, why the authors choose the multiple polynomials with even orders as critic neural network basis functions for simulation?
For example: basis function is given as $\phi (x)$=[x1^2, x1x2, x2^2, x1^4, x1^2x2^2, x2^4 ]', x=[x1, x2]'
Dear Wei Xin ,
In reinforcement learning, it is a common practice to map the state(-action) space to a different one using basis functions. This transformation aims to represent the input data in a more informative form that facilitates and improves subsequent steps. As a “good” set of basis functions result in better solutions and defining such functions becomes a challenge with increasing problem complexity, it is beneficial to be able to generate them automatically. In this paper, we propose a new approach based on Bellman residual for constructing basis functions using cascadecorrelation learning architecture. We show how this approach can be applied to Least Squares Policy Iteration algorithm in order to obtain a better approximation of the value function, and consequently improve the performance of the resulting policies. We also present the effectiveness of the method empirically on some benchmark problems.
https://hal.inria.fr/hal-00826054/document
Regards,
Shafagat
Hi,
It is important to know that the approximation of the functions is done either by polynomials, Fourier series (linear combinations)... or by neural networks (non linear function) using the gradient and in particular NNs by the back propagation technique. Why the even order is to make appear during the development of the gradient or the retro propagation the two terms related to the parameters/sub functions.
Also please take look at the links.
https://tel.archives-ouvertes.fr/tel-00003985/document
Article Optimal Reinforcement Learning-Based Control Algorithm for a...
Article Reinforcement learning-based optimised control for a class o...
Best regards
It can make the control algorithm significantly simple to compare with the existing optimal control methods to achieve the optimized control.
- Badges
- Science method
- Similar topics
- Methodology