Least squares is a special case of an optimization problem. The objective function is the sum of the squared distances.The solution can be found analytically.

Gradient descent is an algorithm to construct the solution of an optimization problem approximately. The benefit is that it can be applied to any objective function, not just squared distances.

So if you find the least square result unconvincing, squared distances are maybe not the adequate objective function for your problem. If you find an alternative one, gradient descent is a way to obtain at least a locally optimal solution.

I recommend these tutorials

https://towardsdatascience.com/linear-regression-simplified-ordinary-least-square-vs-gradient-descent-48145de2cf76

https://www.youtube.com/watch?v=IyDwQNXDWns

https://www.youtube.com/watch?v=wjhT2YFNmlw

LS is one of the method of data smoothing. You also can use interpolation, splain, etc. approaches. To find optimal point, you also can use different methods. One of them is gradient descent. But... If you task (e.g., LS) have a lot of local minimums, gradient descent works bad. In this case you should use other algorithms - genetic, cross-entropy, tabu, etc.

Wenbin Wang Gradient descent basically find optimal parameters in small step. Moreover, it's more general approach you can apply to any optimization problem but it's slow as it optimizes in small steps. Whereas Least Square deals with specific type of optimization i.e. Sum of Square. Coming to key point it depends on which type of data you are fitting as best to my knowledge as highlighted by others also gradient descent can be a good choice as it's general.

Thomas Neusius Cristian Ramos-Vera Sergey Porotsky Siraj Munir Thank you, all.