Naresh Bhimchand Yes, you can use the gradient descent approach to minimize the values in a 30 x 1 matrix. The gradient descent technique is a prominent optimization approach in machine learning and deep learning for determining the best values for model parameters.

Gradient descent works by iteratively updating the parameters in a way that decreases the goal function. Typically, the objective function is a cost function that assesses the difference between the expected and actual values in the data. The cost function's gradient with respect to the parameters is determined and used to update the parameters in the direction of the steepest descent.

To use gradient descent on a 30 x 1 matrix, first establish the objective function and compute its gradient. The parameters would then be initialized and repeatedly updated using the gradient until the cost function converged to a minimum.

Here's a simple gradient descent pseudocode:

1. Set the parameters (theta)

2. Calculate the cost function (J) and gradient (dJ/dtheta).

3. Repeat until convergence is reached:

a. Recalculate theta = theta - alpha * dJ/dtheta

b. Determine the cost function (J) and gradient (dJ/dtheta).

The learning rate, alpha, is a tiny positive variable that sets the step size for each update.

Check out the following links for further information about gradient descent and its implementation:

• Andrew Ng's Coursera course on Machine Learning (https://www.coursera.org/learn/machine-learning)
• An Introduction to Mathematical Optimization by Floudas and Pardalos (https://www.springer.com/us/book/9780387987566)
• Neural Networks and Deep Learning: A Textbook by Charu Aggarwal (https://link.springer.com/book/10.1007/978-3-319-54184-9)

Hi Qamar, Thanks

Gradient descent is a heuristic. To answer there is no guarantee that a solution around the optimal point will be found. However it still does a good job.

I recommend Wayne. L Winston "Operations Research: Applications and Algorithms".

For more stuff related machine learning the Udemy course "Data Science & Machine Learning(Theory+Projects)A-Z 90 HOURS" is good

Ok thanks for your feedback Tegh Singh , From the above problem, which type of Optimization algorithm will be useful to find a optimized result. Because, i tried with Gradient descent and then, with Differential evolution algorithm but i am not getting the clear solution from the problem.

It depends on the cost function and the model that you are using. Gradient descent will converge to the optimal value (or very close to it) of the training loss function, given a properly set learning rate, if the optimization problem is convex with respect to the parameters. That is the case for linear regression using the mean squared error loss, or logistic regression using cross entropy. For the case of neural networks with several layers and non-linearities none of these loss functions make the problem convex, therefore there is no guarantee that you will find the optimal value. The same would happen if you used logistic regression with the mean squared error instead of cross entropy.

An important thing to note is that when I talk about the optimal value, I mean the value that minimizes the loss in your training set. It is always possible to overfit, which means that you find the optimal parameters for your training set, but those parameters make inaccurate predictions on the test set.