Alan, gradient methods require a mathematical model (or an estimate) of the objective function, which isn't available in many real world problems. Also, gradient-based methods can get stuck in local optima. So, GAs are more likely to find the global optimum, and require relatively little knowledge of the problem being solved. However, they may be less efficient for simple problems, and other methods may be better when you know more about the structure of the solution landscape. Given that you have multiple constraints, I would suggest you also look into multi-objective methods, such as NSGA-II, which have been widely applied to difficult engineering problems.

GA are part of metaheuristic algorithms, ie, they can find a good near-optimal feasible solution in a reduced computational time, but it is not usually the global optimal solution. Specifically, in GA you need to know the objective function and make a smart representation of solutions using binary strings. Also, a crossover function have to be provided. The most critical things of GA are a faster computation of objective function and how to do good crossover operation in order to avoid unfeasible solutions.

The objetive function can be any function (convex or non-convex; linear or non-linear). In fact, it can be computed by a deterministic algorithm instead a mathematical function.

There is a great discussion for a similar question at RG:

https://www.researchgate.net/post/Why_is_back-propagation_still_used_extensively_to_train_ANN_while_it_is_beaten_by_GA

 Gradient-based methods can get stuck in local minimum and their performance are dependent on initial values of design variables.

Your question is specific to optimal design of steel structures and as a structural engineer, I can say the following:

In structural optimization problems,the variables are often discrete and moreover the constraints (stress, displacement, etc) are not explicit. Constraints are only available in an implicit form and must be obtained from FEM analysis. That is why evolutionary algorithms (GA, PSO, ACO, DE, ...) are more suitable for the optimization.

I hope that answers your question.

Many thanks for the feedback everyone. Much appreciated. Behrouz - the implicit nature of constraints and discrete nature of variables makes clear EA is suitable vs traditional methods

Dear Alan,

I agree with the previous answers. With a gradient method tue solution is depending on the initial guess. Hence, if you have more local solutions, it is difficult to find the optional one. Think, for example, to a sinusoidal function in which you have infinite stationary points.

In any case, before to start with an heuristic tecquiniqe, like a GA, try to explore the domain of your problem: maybe you are able to restraint the domain, by using your head and not an artificial intelligence method.

Regards to all and good work

Manuel

Hi all,

I agree with all previous comments. Probably the capital advantage of GAs is that you don't have to worry much about explicit definitions. To avoid getting stuck in local optima, I have always tried with different parameters for the GA (size of population, times, rates of mutation, etc) as well as different initial subsets. Furthermore, it's always a good strategy to perform any sort of local search around the (a priori merely local) optimum found by GA, just to make sure that there are not any other "better" solutions in the surrounding region of possible solutions.

Best,

Good point Emili. This approach is nowadays referred to as a 'memetic algorithm' and has become quite popular in applied research as a way of balancing the benefits of population and local search algorithms.

The gradient based methods require to satisfy certain mathematical properties such as differentiability, continuity and convexity - that may not be the case for many real world problems. Genetic or Evolutionary algorithms do not require to satisfy such properties. However they can be used even for the problems that satisfy these properties. So one can apply Genetic and Evolutionary algorithms as general purpose algorithms for all types of problem. I agree with Michael about the memetic version of the algorithm - that enhance the performance significantly.

The textbook description of genetic algorithms is that they sometimes yield a local optimum instead of a global optimum. They were first described (to my knowledge) by C.E. Shannon in his thesis.

So for a problem which has nice mathematical formulations, you would be better off finding the optimum mathematically.

In sociology, when modelling complex systems, it is sometimes better to use a heuristic method since the problems are not well defined (parameters are not well defined or not known). In sociology, sometimes, the behaviour of the system is to be in a non-optimal state, so for simulation purposes, it is a good idea to use heuristic methods such as GA's.

Basically, in some cases, the equation describing the system is hard to find.

If your problem does not have local optima, then GA's may still be good over other methods. Even if the problem has many local optima, GA's could still be interesting if the problem is not well-defined.

In any cases, the method is heuristic and may yield non-optimal solutions. If the hard optimum is needed, then GA's are usually not used, unless the hard optimum cannot be found mathematically.

In heat transfer problems, good estimates of thermal properties are obtained by using an efficient hybrid method which combines a genetic algorithm with a gradient based method (the Levenberg-Marquardt method). This combination aims to search, with the genetic algorithm, for a suitable initial-guessed solution for starting the optimization with the gradient based method.

If you want to decide, which optimization method to use for your problem you first need to consider the information that is available. Most important are:

1. Is the problem bound constrained or are there more complex structured constraints?

2. Do you have an explicit formula for the objective function and all constraints?

3. Is your function smooth (twice continuously differentiable), differentiable, or non-smooth?

4. If not, do you have gradients available (analytical not numerical), or even Hessians?

5. Is your objective function cheap (can do hundreds of function evaluations in a second) or expensive (one function evaluation takes a significant time - tenths of seconds up to minutes)?

6. Is your objective function convex?

7. What is the dimension of your problem?

If you cannot easily answer questions 3 or 6 never mind, then assume that the answer is no.

Now you can start choosing an algorithm. If you have complex constraints, then likely GA is not for that problem, since most GA methods have problems with constrained problems.

If you have analytical formulae for all functions involved and the dimension is not too high (< 50) then a deterministic method (BARON, LINGO, etc.) is likely the best choice. By algorithmic differentiation in case of analytic formulae the answer to 4 will be yes then.

If you have gradients or Hessians it is worth to note that every global minimum is a local one, and local optimization is much more efficient than every black-box method if gradients or more are available. In that case some sort of globalization scheme (randomization, pure branching, Krieging, etc.) together with local optimization (SQP methods, IPOPT, KNITRO, WORHP, etc.) is likely the most efficient approach.

If you have non-smooth functions and subgradients available, then some sort of bundle method can replace the local optimizer (e.g. MPBNGC or LMBM).

If you do not have gradients you cannot use a gradient based method (numerical gradients need too much effort). Then questions 5, 6, and 7 become extremely relevant.

If you know that the function is convex you should use a method specialized for convex black box optimization.

Otherwise, 5 gives a very important information. If your objective function is cheap some evolutionary method (e.g. CMA-ES) is a very good choice. If it is expensive, I would go with a DFO method (e.g. MCS, LGO, SNOBFIT, TOMLAB/GLCCLUSTER, TOMLAB/MULTIMIN, etc.), depending on the dimension.

If your function is very high dimensional, expensive, and black-box you can try whatever you like, as there are no methods which are satisfactory in that situation, and any of them can come up with a good solution.

I agree with Hermann Schichl.

i agree with Behrouz

Thank you again to all contributors. I will make use of these responses in my work and any subsequent write-up. Alan

Thank you Naouel Daouas for clarifying.

Gradien methods has many requieriments such derivation. In case of GA,they do not make any assuption about the objective function.

This is the most important characteristics of the metaheuristic approaches

All the answer are great

One intersiting advadtage of GA is that they does not use the deterministic definition of a objective functio, so that t can be used for optimization of black models

Advantages of GAs compared to conventional methods:

1. Parallelism, easily modified and adaptable to different problems

2. Inherently parallel; easily distributed

3. large and wide solution space search ability

4. non-knowledge based optimisation process used of a fitness function

for evaluation

5. Easy to discover global optimum and avoid trapping in local optima

6. capable of multi-objective optimisation can return suite of

potential solutions

7. Good choice for large scale/wide variety of optimisation problem