I am working on the optimizers of neural networks. It is a very important issue to calculate the gradient to find the global minimum for a loss function to ensure the adaptability of an optimizer. Generally, we calculate the first-order derivative in this case. To get a more convergent equation (objective function), how do the fractional derivatives play a role? What are the impacts of fractional calculus in this domain?