Article Meta-analysis of deep neural networks in remote sensing: A c...

Based on your explanation, I don't think meta-analysis to compare the behavior of evolutionary optimization algorithms on well-established benchmarks will be possible. If there is no studies directly comparing between both groups, one alternative is to perform a network meta-analysis, provided that the studies evaluating optimization algorithm and the studies evaluating the well-established benchmarks are controlled by placebo and/or other similar controls. However, this may not be the case as you stated that there are no control groups.

Another alternative is to perform a single-arm meta-analysis to pool the studies evaluating the behavior of evolutionary optimization algorithm. However, you may only be able to get an approximation of the efficacy of such algorithm without being able to compare them with the established benchmarks.

I hope this helps.

Julio. You ask a very important question. In my experience benchmarks in any computing field are narrowly focused on a best solution in comparison to other solutions. Such methods fail to consider that good or best solutions may be found accidentally, by solution engineering, or other influences. I have some papers and patents on "Benchmarking Combinatorial Problems and Algorithms for Their Solutions that perform a statistical comparisons by determining the mean and standard deviation of solutions to a given problem using a given method. You will find that some algorithms or methods have a better mean, or wider distribution. But the best measure is to compare the solutions found at a fixed distance from the mean, measured in standard deviations. Many of my publications are available through Research Gate. I am working with two individuals in the EU who are re-evaluating their library of benchmark problems and the results reported in the literature. Evolutionary algorithms are particularly hard to compare, but I have found that a good method is to compare the average solution for each of a sequence of predetermined generations. In many cases a an evolutionary algorithm follows the path of sampling a Gaussian distribution. Let me know if you would like to discuss and I will set up an on-line meeting. I can send you some of the graphs that result from these methods and you will see these methods provide considerable insight.

https://scholar.google.com/citations?user=XOg2QnoAAAAJ&hl=en