https://courses.csail.mit.edu/6.006/oldquizzes/solutions/quiz2-s2011-sol.pdf

Dear Shaik

In general you need to trace the surveys for the specific problem. It should include comparisons between the different algorithms performance. Decisions should be made guided by the previous work  done.

Good Luck.

Prof. S. EL-Rabaie

Dear Shaik

As per No Free Lunch Theorem (NFL) [wiki]

In 2005, Wolpert and Macready themselves indicated that the first theorem in their paper "state[s] that any two optimization algorithms are equivalent when their performance is averaged across all possible problems".[3]

It will depend on the type of the problem. Type of the constraints etc. The choice of algorithm will be very much domain dependent.

Regards

Saptarsi

Understanding the problem and having good knowledge of the subject area where it applies are necessary and sufficient to guide your choice of approach.

I may be old-fashion, but it seems to me that 2017's science doesn't have an answer to your question. For example, sustainable development is a problem, and we don't have any algorithm to solve it. What I want to say is that before asking for the best algorithm, first define formally your set of problems to solve ;-)

First, thanks a lot for very interesting and important question!

1. Sometimes, if we know exact solution for our task with low dimension (i.e., 2 parameters instead of real 20 parameters), we can compare different algorithms under "Low dimension task". But sometimes algorithm is good for low dimension and bad for large dimension.

2. Sometimes we can check stability. For example, for random search orienteered algorithms (i.e., GA, CE, Simulated Annealing, etc.) we can change initial seed.