If deep learning is convex, then training will be much easier. No local minima and everytime a global optimal is reached.
Dear Sing Kuang Tan ,
Similarly, many variations of stochastic gradient descent have a high probability of finding a point close to the minimum of a strictly convex function. Deep models are never convex functions.
https://www.kaggle.com/general/325028
Regards,
Shafagat
I have shown that deep learning can be converted to discrete Markov random field, then to convex optimization. Therefore I think it is convex. https://vixra.org/abs/2112.0151
No, deep learning is not inherently convex. Convexity refers to the property of a mathematical function, where any two points on the function lie below or on the line segment connecting them. Convex optimization problems have a unique global minimum, making them easier to solve.
Deep learning, on the other hand, typically involves training neural networks with multiple layers, and these networks are highly non-convex. Non-convex optimization problems have multiple local minima, making it challenging to find the global optimum.
While it's true that deep learning training can be challenging due to the non-convex nature of the optimization problem, researchers and practitioners have made significant progress in developing algorithms and techniques that allow us to train deep neural networks effectively and achieve good performance on a wide range of tasks.
- Badges
- Science topic