Generally speaking, the indirect method obtains the optimal solution by directly solving the optimality conditions (obtained from the maximum principle) of the optimal control problem. The direct method discretizes or parameterizes the optimal control problem into an optimization problem and then numerically solves it to obtain the optimal solution. For many optimization problems, especially those with constraints, the optimal solution is usually obtained by solving the optimality conditions of the optimization problem. For instance, the well-known nonlinear programming algorithm SQP has been proven to be equivalent to solving the optimality (KKT) conditions of the original problem using the Newton method. There are studies indicating that under certain conditions, the necessary conditions of the discretized optimization problem and the optimality conditions of the original optimal control problem are equivalent. Therefore, in most cases, both the direct method and the indirect method obtain the optimal solution through optimality conditions. From this perspective, there is no fundamental difference between the direct method and the indirect method.
A large number of optimal control problems are too complex to be solved analytically. These optimal control problems can be solved numerically. Both the direct methods and indirect methods are numerical methods for solving optimal control problems.
In Indirect methods, we rewrite the optimal control problem (which is an optimization problem) in another optimization problem. In our case, the new optimization problem is a Boundary Value Problems (BVPs) which are based on Pontryagin's minimum principle. This BVP INDIRECTLY represents the original optimal control problem. Few indirect methods are shooting methods to convert BVP to IVP, collocation methods like Euler, Simpson, Trapezoidal to use root finding techniques, the Forward-Backward Sweep Method (FBSM) using Newton methods and RK methods. Here, BVP is still a continuous function which is discretized for solutions. Also BVP need not give minimum solution. It could be a saddle point also. Solving the HJB only tells whether it is a minimum.
On the other hand, direct methods DIRECTLY discretize the control and state variables and directly solve for the optimal values. Here we first discretize the Optimal control problem and construct optimality conditions given by the KKT equations. Steepest Descent is an example of a direct method where the design variables are directly manipulated to minimize the objective function.
In comparison, direct methods like steepest descent are easy to implement and computationally less expensive. However they converge slowly, especially for the complex objective functions. When it comes to indirect methods, solving a BVP becomes challenging as most of the solvers and Newton algorithms are sensitive to the initial guess. In addition, indirect methods need the additional information of derivatives, such as gradients or Hessians.
In recent times, researchers are creating hybrid methods where the result from the direct method is used as a starting guess to improve the convergence of the indirect method. Also, softwares like GPOPS-II are gaining prominence where the continuous-time optimal control problem is solved by approximating it as a sparse nonlinear programming problem (NLP).
D Bhanu Prakash Thank you for answering my question.
In my opinion, the discrete boundary value problem (BVP) and the KKT equations are equivalent. The co-state variable in BVP should also be equivalent to the Lagrange multiplier in the KKT equation. The solution of the KKT equations may also be a saddle point. The initial guess of the Lagrange multiplier is also significant for solving the KKT equations. Many solvers include a generator to generate the initial values of Lagrange multipliers. Numerical tests show that these generators are very effective and robust. I think these generation methods can also be used to generate the initial value of the co-state of the BVP (it does not mean transferring the results obtained by the direct method to the indirect method).
I don't know if I expressed myself clearly.
- Badges
- Science topic
- Similar topics
- Mathematics