Hello

The choose of the step size is a crucial part for any optimization problem. In fact, if you are trying to minimize a function F, using a gradient-descent based method ( which is the case for optimizers used in machine learning in general) you need to choose a step size smaller then then Lipshitz constant of the gradient of F to be sure about the convergence of your optimization scheme. In machine learning, we normally don't have access to the analytic form of the gradient of the loss function, thus we don't know analytically this Liphshitz constant, this is why we choose this constant.

You can read those slides for more intuition about optimization in genera and about l the choice of the step size.

One of the fundamental operation in registration is interpolation for proper reconstruction hence the step-size is an important metric ...

Article A Review on Medical Image Registration as an Optimization Problem

I'll leave this image file here so you could see visually what step size means for a gradient descent and how it affects the convergence/divergence of your optimizer :

Conference Paper Some Practical Aspects on Incremental Training of RBF Networ...

hi

The choosing the step size is a crucial part for any optimization problem.

if you are trying to minimize F function , you can use gradient-descent based method ( which is the case for optimizers used in machine learning in general)

you need to choose a step size smaller then then Lipshitz constant of the gradient of F to be sure about the convergence of your optimization scheme.