As a core idea of gradient boosting problems is related with functional gradient descent algorithm and the solution implies iterative addition of a new model that learns on previous pseudo residuals, then probably it makes sense to transfer somehow the problem into the ordinary equation form (or even, in PDE if it possible formally). The are 2 questions? Does it make sense? Were there any efforts to test this approach? Thank in advance!
I do not know if that is exactly what you are looking for, but there is a seminal paper on that matter: "A Differential Equation for Modeling Nesterov's Accelerated Gradient Method: Theory and Insights", by Su, Boyd and Candès. There, they study the accelerated gradient method by analyzing the associated differential equation.
Juan Jose Maulen Muñoz thanks! Yep, it looks like what I was looking for. I haven't gone in depth yet. But it seems there is no functional gradient descent mentioning. I'm more specifically interested in ODE/PDE representation over functionals.
- Badges
- Science topic
- Similar topics
- Mathematics