In cases where meta-heuristic optimization algorithms, such as the Particle Swarm Optimization (PSO), have to be evaluated, the most common performance metric is the total number of objective function evaluations (NFE), combined with the value of the objective function at the global optimum. Other than that, another metric is the time needed for completing the task, but that's directly proportional to the NFE metric.

In cases where the optimization problem turns out to be more tedious to solve, there is a chance that the implemented optimization algorithm won't always reach the global optimum, if tested on a finite number of independent trials. The percentage of the successful runs over the sum of all attempted runs could be a good performance metric in that respect.

For more information check:

1. Jia, L., Gong, W., Wu, H., «An Improved Self-Adaptive Control Parameter of Differential Evolution for Global Optimization», Z. Cai et al. (Eds.): ISICA 2009, CCIS 51, pp. 215–224, 2009. (Springer-Verlag Berlin Heidelberg 2009)

The performance of a meta-heuristic algorithm like Genetic Algorithm, Prey-Predator Algorithm and Particle Swarm Optimization, you can use statistical analysis like hypothesis testing after running the algorithm with a fixed algorithm parameter a number of times, say 50 times.

For instance if you want to use hypothesis testing, your null hypothesis will be the average maximum functional value for an optimization problem (maximization problem with maximum value say 1) is at least 0.99 with 95% level of confidence.

I have a paper in press on this aspect on International Journal of Information Technology and Decision Making with title "Prey-Predator Algorithm: A new meta-heuristic optimization algorithm". (If you are interested I can send you a copy when it is published)

Dear Surafel, I want this paper. This method to analize the performance of a metaheuristic you described is very interesting, already know to me, but never used in my papers.

Dear Tarciso

find below the link to the paper

https://www.researchgate.net/publication/262858727_PREY-PREDATOR_ALGORITHM_A_NEW_METAHEURISTIC_ALGORITHM_FOR_OPTIMIZATION_PROBLEMS

Article Prey-Predator Algorithm: A New Metaheuristic Algorithm for O...

In addition to the previous posts about the matter, the following paper summarizes the statistical methods used to assess the merit of evolutionary and swarm metaheuristics.

Derrac, J., García, S., Molina, D., Herrera, F., «A Practical Tutorial on the Use of Nonparametric Statistical Tests as a Methodology for Comparing Evolutionary and Swarm Intelligence Algorithms», Swarm & Evolutionary Computation 1 (2011) 3-18.

You can also have a look at the tutorial "Experimental Evaluation of Heuristic Optimization Algorithms: A Tutorial" which you can find from http://akturk.bilkent.edu.tr/ie573/local1.pdf

Dear Souza Costa,

You can count the number of times that your algorithm gets stuck in local minimum points. The repetitive solutions I mean.

Hope you find it useful.

Farshid Keivanian

MSc in Electronics Engineering

University of Birjand

Iran