Consider line search and search for the two Wolfe conditions.

Hello. Assuming the iteration is stable, in the sense that Nikolay has mentioned above, I don't think you can guarantee convergence unless you have very specific conditions on your system and initial conditions. This is the nature of stochastic optimisation. It might be possible to characterise the average convergence time, but again it depends on the actual estimation problem, the initial conditions and the step size. If you have a small enough problem, maybe you can do some simulations to understand the convergence conditions better.

For a RG discussion of some possible significance for those interested at this topic: https://www.researchgate.net/post/Recursive_Least_Squares_can_anyone_explain_to_me_what_exactly_this_is_or_where_can_i_find_info_about_it