Stochastic Gradient Descent is a better solution for real-world large-scale problems. It is widely used as the optimization algorithm in large-scale machine learning methods like Deep Learning. For instance, it can be used in real-life machine learning problems like linear regression.

@fabrizio

Thanks for your answer. In fact, I am looking for real-life problems where to apply these technics you mentioned like ML, because ML itself is more of a theory. I already worked on the optimization of a rocket's launch and it was quite interesting to learn about aerodynamics and connect it to the convex analysis and other Mathematical tools.

You can look into Multivariate Calibration and Prediction methods used as part of Chemometrics discipline. Here you can apply deep learning methods such as ANN, which uses gradient descent methods for network parameters optimization, on chemical datasets to achieve tasks of calibration and predictions.

Perhaps: Classical (Markowitz) portfolio optimization (and all its variations) is an easy application of convex optimization, with a lot of use in the Fintech arena in nowadays.

Convex optimization theory has an importent aspect: duality gap. This gap is occurred for bad constraints. To search for such problems, we use the analysis of orders of smallness of infinitesimal quantities