Are there optimization problems that require and use partial derivatives higher than the 2nd order? I.e. 3rd, 4th, 5th order partial derivatives used for and during optimization, particularly convex or non-linear optimization problems.
Optimization method or algorithms based on calculating the Newton step, typically use (up to) 2nd order partial derivatives.
Dear Brian, I am not a real specialist in the topic, but I am unaware of methods using high-order derivatives. The point seems rather to achieve a good convergence speed with a lesser amount of information than to obtain an extremely high speed with a higher amount. As an example, BFGS method is usually preferred to Newton's, despite its slightly slower convergence (in terms of iterations), because it does not require second derivatives, and has a lower complexity per iteration.
Dear Brian,
even it is not common to use high order derivatives I thing it is possible. Indeed, given a singular point it is possible to have the first, the second and the third derivative null while the fourth derivative is not null. In this case the nature of the singular point will be given by the sign of the the fourth derivative. e.g. function power 4...
Hi, I agree with Fotsa Mbogne David Jaures, in some very particular cases when the second derivative is singular it is possible to compute "manually" higher order derivatives to try to find the optimum. This approach is not implemented in existing tools because 1) it would require the objective function to be of classe Ck with k greater than three 2) a quadratic approximation of the cost (or of the Lagrangian for constrained problems like in SQP methods) leads to a quadratic programming problem at each iteration whose solution is easy to compute. When using a third or 4th order approximation would lead to more complex problems to solve at each step of the algorithm.
- Badges
- Science topic
- Similar topics
- Mathematics
- Applied Mathematics