As most people have indicated the best solution depends on the 'surface' you are optimising and the number of dimensions. If you have a large number of dimensions and a smooth surface then traditional methods that use derivatives (or approximations to derivatives) work well such as the Quasi-Newton Method. If there are a small number of dimensions and the surface is fairly sensible but noisy then the Nelder and Mead Simplex works well. For higher dimensions with noise but still farily sensible (hill like) then simulated annealing works. The surfaces which are discontinuous and mis-leading are best addressed with the more modern heuristic techniques such as evolutionary algorithms. If you are trying to find a pareto-surface then use a multi-objective genetic algorithm. So the key things are how many dimensions, is the surface reasonably smooth (reliable derivatives), do you want a pareto surface or can you run multiple single criterion optimisations. The other questions is, do you need to know the optimum or do you just want a very good result. There are often good algorithms for approximations to the best result, for example using a simplified objective function which can be found much faster to get a good rough solution which may be the starting point for a high fidelity solution. Sorry if this indicates it is complex, it really does depend on the solution space. Do not forget traditonal mathematical methods used in Operational Research as well. Good Luck!

It is completely dependent on the type of problem.

Some simulations showed that the BFGS method for large sample size is the best one. However, for finite-sample, this may produce a small bias.

Generally, the situation and the function you are working on, are the important features for the choice of the optimization method.

Hi Taimoor. A good starting point for you is to define your problem and decide which problem class it is associated with. If your problem is linear and in reasonable sizes, I suggest you to focus on exact solution approaches. You may develop a mathematical model and solve it using well-known solvers such as GUROBI or CPLEX.

If your problem is a mixed-integer programming problem, then there are well defined problems in the optimization literature such as assignment problem, traveling salesperson problem or knapsack problem. If your problem fits into the framework of one of these problems then you can focus on the solution approaches (both exact and heuristic) that has been developed for it.

If your problem is NP-complex, you can use a library with the different algorithms and choose the best one in each case

Dear Taimoor,

Assuming that your problem is NP-hard, as the no free lunch theorem claims it, there is no universal best method.

So there is nothing to do except test and test and test.

Because, choosing a method between different one is not the only thing to do :

For each method, there are many parameter values possible and many improvements that could suit with your problem....

You should first do a bibliography related to your problem to check which method is often used in your context.

For testing you can use experimental designs to set representative values for parameter and frameworks which implements many single/multi objectives metaheuristics for discrete/continuous optimization (such as MOEA, JMetal).

Good luck !

For optimisation of power systems, there are some really interesting approaches bases on semidefinite programming and second-order cone programming, in addition to the usual metaheuristics.

It depends to the specific problem you try to solve.

Assalamu alaikum and hello everyone,

Do you want to just get the results for a specific problem or to do proper research?

If you want to get some results then in Electrical Power field Newton Raphson is one of the quickest method provided good starting values are available. However, if you want to do research then genetic algorithm (GA) can be a good start and then there is particle swarm optimization (PSO), differential evolution (DE) and so on.

In my study (harmonics removal in inverters, power electronics) I have found DE to be the best choice after doing a comparison between above mentioned algorithms.

In short do a strong literature review, pick some algorithms which are considered as good, find a new one and then do a proper comparison.

Regards,

Abdul Moeed Amjad

Thank you so much Abdul Moeed Amjad Sir.

@Taimoor Mohsin

My pleasure.

the best way would be to formulate your objective function and find out which mathematical benchmark function is similar one or near to similar. then you can search performance of different algorithms on that particular benchmark function. the algorithm which comes out to be better may perform better on your problem too.

for example if my objective function is somewhat similar to quadratic equation than algorithm performing better on quadratic equation is likely to perform better on my problem too.

remember as per "no free lunch theorem" all meta-heuristic algorithms are relatively equal.

All the best!

Taimoor Mohsin For this purpose yo need to form the objective function of the optimization problem. Then you can decide the optimization algorithm. Generally classical techniques gives more accurate results but these are time consuming and having some limitations. For faster processing you can select any of the meta-heuristic algorithm.

Dear Vikas,

sorry, but in most optimization problems there exist no objective optimization function because most problems are multicriterial ones. Only in problems which can be reduced to one-criterial problems you can have an objective optimization funtion.

There are two families of optimization algorithms: namely, Traditional and nontraditional. The metaheuristic optimizers belong to the latter family. If you are to choose a certain algorithm for your problem, first of all, you need to identify whether the problem is single or multi-objective. Single objective optimization problems are easier to solve compared to the multi-objective ones. Accordingly, there are fewer established algorithms for multi-objective problems.

Secondly, you need to have a close look at the current literature related to metaheuristics or nature-inspired optimizers, as every now and then there are novel algorithms being proposed which are outperforming the older ones.

Then you need to try and implement the newer ones for your problems and compare its performance against the older ones as it will increase your chance of publishing in reputed journals.

One way to implement a novel algorithm is to try out some intelligent mixture of current algorithms to form a hybrid one with the hope to improve the performance of the parent algorithms.

Thank you so much Ashik Ahmed Sir. My problems have multiple objectives that also contradicts with each other. I will try to find the latest algorithms related to my problem that may produce better results. Currently i am working on Genetic Algorithm.

Sorry, I must know the mathematical formulation of your problem so that I can answer usefully.

It seems you have a problem that can be solved only by heuristic methods. And the most popular algorithms are GA, PSO and ACO.. There are many variations and hundreads of other algorithms too.. Two things one may consider i. e.. Time and degree of acceptance.. real time problem or not....

Hi,

Sorry, there lots of methods to solve an optimization problem. It all depends on your objective function and constraints form, whether they are convex or not, then decide how to deal with it. However, you can see similar works and find out which one is more suitable for solving your optimization problem in aspects of complexity and convergence.

As most people have indicated the best solution depends on the 'surface' you are optimising and the number of dimensions. If you have a large number of dimensions and a smooth surface then traditional methods that use derivatives (or approximations to derivatives) work well such as the Quasi-Newton Method. If there are a small number of dimensions and the surface is fairly sensible but noisy then the Nelder and Mead Simplex works well. For higher dimensions with noise but still farily sensible (hill like) then simulated annealing works. The surfaces which are discontinuous and mis-leading are best addressed with the more modern heuristic techniques such as evolutionary algorithms. If you are trying to find a pareto-surface then use a multi-objective genetic algorithm. So the key things are how many dimensions, is the surface reasonably smooth (reliable derivatives), do you want a pareto surface or can you run multiple single criterion optimisations. The other questions is, do you need to know the optimum or do you just want a very good result. There are often good algorithms for approximations to the best result, for example using a simplified objective function which can be found much faster to get a good rough solution which may be the starting point for a high fidelity solution. Sorry if this indicates it is complex, it really does depend on the solution space. Do not forget traditonal mathematical methods used in Operational Research as well. Good Luck!

Try four to five algorithms based on single and multi objective and compare their results to find the best one or the one that is better than others in some perspectives.

Think about the problem, you would like to solve. Then, make a model, with appropriate objective function(s) and constraints. It seems, that your problem involves "time" as well, and maybe there is uncertainty involved as well. You soon might find out, that the model you end with, is too complicated, although being complicated is also for a part in the eye of the beholder! There seems to be no easy way out of your predicament! So, you need time, and hard study! And, there is not one good model! In any case, never use one model!!! Good Luck!!

I would suggest you do the following:

1. First and foremost, focus on formulating a parsimonious mathematical programming model of the problem at hand — ‘parsimonious’ meaning ‘simple but complete’, and to that end, the ‘Partial Relaxation Algorithm’ described in the paper attached might come in handy.

2. Next, try out a ‘general purpose’ solver and see whether the solutions you get are satisfactory; this is where evolutionary algorithms such as GENO would play a part.

3. If you are not satisfied with the results at Stage 2, analyse the model from Stage 1 for any special mathematical structure and seek out specialized algorithms whose design exploits the specific structure(s) found, e.g. Quadratic objective function with linear constraints; run the algorithms to see there is significant improvement in performance.

4. If not happy with results at Stage 3, you may have to revisit Stage 1.

Good luck!

Apparently, I cannot edit my previous answer, so I will re-submit:

1. First, focus on formulating a parsimonious mathematical programming model of the problem at hand — ‘parsimonious’ meaning ‘simple but complete’ in the sense that it is tractable and includes all the relevant variables. The ‘Partial Relaxation Algorithm’ described in the paper attached might be useful.

2. Second, try out a ‘general purpose’ solver and see whether the solutions you get are satisfactory—this is where evolutionary algorithms such as GENO would come into play; however, the step is optional.

3. If you are not satisfied with the results at Step 2, analyse the model from Step 1 for any special mathematical structure and seek out specialized algorithms whose design exploits the specific structure(s) found, e.g. Quadratic objective function with linear constraints; run the algorithms to see there is significant improvement in performance.

4. If not happy with results at Step 3, revisit Step 1 for further improvements / simplifications of the model.

Good luck!

Using Algorithm depends on the following parameters

1. Length of input elements

2. Time for solving the problem

3. Space consumed for implementing the algorithm

Based on these three parameters you can choose between Greedy, Dynamic, Divide & Conquer and Backtracking approach.

the best way to choose the optimization algorithm is, first you have to decide whether it is single objective or multi objective optimization. and then you have to proceed. suppose you have single objective means Taguchi TECHNIQUE MAY BE USED.

If it is Multi Objective means - Grey Relational Analysis, Response Surface Methodology, Desirability Function Analysis, TOPSIS, VIKOR, may be used.

Mohsin actually has , implicitly, two optimization problems in mind. The first one is related to his problem at hand, a power system, and the second problem is, to find the best algorithm for this problem. Both problems are ill posed in the sense, that we don't have a real clue about constraints and objective functions of both problems! And nevertheless we are trying to help! That is, given the status of Mohsin's problems, really amazing!

Choosing a particular algorithm is dependent on the nature of the problem based on its parameters.

Hi,

To successfully choose the optimization method for a given problem, it is absolutely necessary that the specifications and the problematic must be described in a clear and unambiguous manner also the analysis will focus on the type of design space sought . Relative to a given problem it is defined by the types of responses and their dimensioning as well as the types and ranges of design variables.

For more details and information about this subject, i suggest you to see links ans attached files on topic.

https://mech.iitm.ac.in/nspch52.pdf

https://www.sciencedirect.com/topics/engineering/solving-optimization-problem

http://www.scielo.org.mx/scielo.php?script=sci_arttext&pid=S1665-64232018000500357

http://www.it-weise.de/projects/book.pdf

Best regards

First,simulate your algorithm and compared with other author' algorithm ,if your algorithm results is better,so you can approve your algorithm in your calculations.