In other words which method is not sensitive method for the initial assumption for the input ? Secondly can I use it to minimize function such as F=(x,p,u1,u2) with two inputs to minimize u1,u2 at the same time (by using steepest descent method)
Following Marek's comments, perhaps Ahmad's goal is to minimize F, x being states, p being (fixed and known) model parameters and u = [u1, u2] being the degrees of freedom. I agree that the formulation is pretty much misleading and the following is under the assumption that my interpretation is the good one :)
If your goal is to avoid to any cost local optima, then global optimization methods are suited: they are 'global', that is they converge globally (from any initialization point) to a global minimum . Local optimization methods (like gradient-based methods) have many other advantages, but they are not, of course 'global'. Of course, they can be globally convergent (convergence to a local minimum regardless the initialization point).
There are two main families for global optimization that are stochastic and deterministic methods, respectively. Stochastic methods are often (not always) 'easier' to apply (loosely speaking) but typically cannot guarantee global convergence in finite time. Deterministic methods (such as branch and bound, e.g.) exhibit global convergence in finite time. In my opinion, they are more elegant and theoretically sound, but this is a matter of taste :). However, they are much more involving in terms of maths and computational time. They require other assumptions (C2 differentiability, e.g.) and suffer from the curse of dimensionality more than stochastic methods, on the other hand.
The choice should be, as always, guided by the real needs and the available time for the resolution of the prolem.
I hope this helps!
That depends on what do you call a "success". If reaching the nearest minimum is all you want - then the answer is yes. When your function is unimodal then even very poor start point (located very far from the true solution) will do.
The answer to your second question is negative, since the set "p" (I understand this is the set of searched parameters) will be generally different for minimal u1 and for minimal u2. Look for "multiobjective optimization" instead.
Well, the "banana function" (Rosenbrock's function) I think at least illustrates that the path taken with the steepest descent method is far from the shortest. Depending on the general properties of the problem studied the answer I think can be a yes or a no.
Following Marek's comments, perhaps Ahmad's goal is to minimize F, x being states, p being (fixed and known) model parameters and u = [u1, u2] being the degrees of freedom. I agree that the formulation is pretty much misleading and the following is under the assumption that my interpretation is the good one :)
If your goal is to avoid to any cost local optima, then global optimization methods are suited: they are 'global', that is they converge globally (from any initialization point) to a global minimum . Local optimization methods (like gradient-based methods) have many other advantages, but they are not, of course 'global'. Of course, they can be globally convergent (convergence to a local minimum regardless the initialization point).
There are two main families for global optimization that are stochastic and deterministic methods, respectively. Stochastic methods are often (not always) 'easier' to apply (loosely speaking) but typically cannot guarantee global convergence in finite time. Deterministic methods (such as branch and bound, e.g.) exhibit global convergence in finite time. In my opinion, they are more elegant and theoretically sound, but this is a matter of taste :). However, they are much more involving in terms of maths and computational time. They require other assumptions (C2 differentiability, e.g.) and suffer from the curse of dimensionality more than stochastic methods, on the other hand.
The choice should be, as always, guided by the real needs and the available time for the resolution of the prolem.
I hope this helps!
Steepest descent is a local optimizer. As such it will converge to the closest minimum to the initial guess. If your problem is convex, you can use other alternatives with better convergence properties (quasi-Newton methods, truncated Newton, BFGS based methods, etc).
If you are facing a multimodal problem, i.e. with several minima, then a global optimizer would be more suitable. In this respect, there are several possibilities, deterministic, stochastic, hybrid.
To decide on the type of method, you should take into account: convexity, size of the problem in terms of number of decision variables, presence of constraints, possibility of computing gradient/ Hessian, etc.
Yes. And no.
This question is like asking: Can I use a hammer to build something?
Steepest descent is one of many tools that might provide a local minimum, or not. None of these tools always works. None of these tools never works. To learn more about SD, etc., try to construct some functions that defeat the method, then look up some well known ones. This will increase your wisdom for how to use these tools.
A shorter answer: go ahead and try. Maybe you will be lucky.
There is no relation between selection of a poor or good initial solution, and finding a poor or good solution. What matters is that what path your logic is following. In case of steepest descent method, we cannot guarantee about the result quality. You need to use some reliable method that can guarantee to some extent.
Hi there,
I was looking for a publicly available code for steepest descent method for minimization and I came up the web site whose address is
http://www.math.colostate.edu/~gerhard/classes/331/lab/newton2d.html .
I am running the same code structure by implementing the function,
f(x,y)=(1-x)^2+100*(y-x^2)^2
which is known as Rosenbrock’s “banana” function.
I removed the "&n
- Badges
- Science topic
- Similar topics
- Mathematics
- Applied Mathematics