dear Akalin,

i don't know anything about the steepest descent method, but it very clear when comparing the web site code and yours we can say that the problem is in the gradf formula. 

i don't know how you can change the formula but  i think that its gradf formula that makes  gradient values and others go to infinity after a few iterations.

dear Aklain,

I found this matlab code that could help you:

http://www.wiley.com/legacy/products/subject/engineering/venkat/Chapter6/SteepestDescent.txt

Dear Okk,

 The problem is exactly what you mention. To be able to deal with this problem, I have to put a constrained region on which the code works without going beyond the region that is out of domain, including the minimum point I want to hit.

The codes you addressed somehow does not work. As I understand, it needs to read another Matlab file that we actually  do not have.

Thanks for your help.

I'm not sure, but I notice that you have the step:

x=x_iterated; % update x

Shouldn't this be something more complicated?  I would say, if x_iterated has a smaller cost than x, then move there, but if not, then either quit or decrease the step-size.

n = 0;

tol = 1;

a = 10e-5;

x1 = -3.0;

x2 = 2.0;

x = [x1;x2];

while tol>10e-10

gradf=[400*(x(1))^3 - 400*x(1)*x(2) + 2*x(1)-2 ; -200*(x(1))^2 + 200*x(2)];

tol=abs(gradf(1))+abs(gradf(2));

x_iterated=x-a*gradf;

x=x_iterated;

n=n+1;

end

n,x,tol,

Dear Letchford,

First of all, thank you for your reply. This code that I posted above is working with the change of the magnitude "a". After I posted first message, I realized that smaller "a" makes sense in terms of reaching the minimum after lots of iterations. 

But, right now, we are dealing with writing lines of code for the same purpose but by Conjugate Gradient Method. The codes for CGM is the following. If you have any suggestion, could you share your opinions?

n = 3;

tol = 1;

a = 10e-20;

x1 = -1.0;

x2 = -1.0;

x = [x1;x2];

% gradf_store=[];

gradf=[400*(x(1))^3 - 400*x(1)*x(2) + 2*x(1)-2 ; -200*(x(1))^2 + 200*x(2)];

gradf_store=gradf;

gradf_store_tr = gradf_store';

S=-(gradf_store_tr(1,:))';

x_iterated=x-a*S;

x=x_iterated;

while tol>10e-10

gradf=[400*(x(1))^3 - 400*x(1)*x(2) + 2*x(1)-2 ; -200*(x(1))^2 + 200*x(2)];

gradf_store=[gradf_store gradf];

gradf_store_tr = gradf_store';

beta=(sqrt((gradf_store_tr(n-1,1))^2+(gradf_store_tr(n-1,2))^2))/(sqrt((gradf_store_tr(n-2,1))^2+(gradf_store_tr(n-2,2))^2));

S=-(gradf_store_tr(n-1,:))'+beta*(gradf_store_tr(n-1,:))';

tol=abs(gradf(1))+abs(gradf(2));

x_iterated=x-a*S;

x=x_iterated;

n=n+1;

display([num2str(tol),' ',num2str(x(1)),' ',num2str(x(2))])

end

n,x,tol,

Sorry, I don't know anything about the CGM.  You could try asking someone from this group:

http://www.maths.ed.ac.uk/ERGO/

This method does not guarantee you the global-optimal value. It will stack in the local-optimum. As far as I know, there are no real recommendations for certain initial values, as before you executed the algorithm you have no idea about the function you explore. Therefore, you can randomly sample initial position. If you are looking for a global-optimum then you would need to search for the algorithms, that can avoid local optima, e.g., VNS or GA. 

If you have an initial state of the problem, the best initial point is the variable values of the current state. otherwise it is better to find it by trail and error. That is, by selecting several initial solution and seeing which one performs better. Otherwise there may not be a general rule for finding the initial guess.

It is a case to case problem. If your problem is a convex problem, then the no matter what initial guess you select, it will converge. If it is not a convex problem than gradient descent method only will give you a local minimum. If you want to obtain the global minimum then god should give you the initial guess around the global minimum.