Let's imagine a generative hyper-heuristics generates very short metaheuristics. No limitation of the length of the algorithms has been given in some inductive bias. Is it really necessary to compare the hyper-heuristics run against an exhaustive search. I am not convinced it is necessary and cannot bring myself to find an advantage. Can you give your opinion please.
Please have a look at the link https://www.researchgate.net/post/How_to_evaluate_the_quality_of_metaheuristic_algorithms_without_any_benchmarks
Hi Patricia,
if I understood correctly, I guess the point here is that, because the generated metaheuristic might be very short, an exhaustive search might actually do quite well. But that depends on the starting point of the exhaustive search itself. Given that no limitation on the length of the solutions has been given, it is reasonable to accept that an exhaustive search might start from a given random point (rather than the shortest possible solution). If you cannot make any assumption on lengths (which you seem to suggest), then this sounds like an acceptable benchmark.
I would use the measure published by Dolan and Moré - the "performance profile". Here is a link to the paper: doi:10.1007/s101070100263
From the abstract:
"We propose performance profiles — distribution functions for a performance metric — as a tool for benchmarking and comparing optimization software. We show that performance profiles combine the best features of other tools for performance evaluation."
Without a benchmark you are still lost, regarding an algorithm's performance. You need to compare with something, and actually to something that is natural.
A heuristic could be compared with the exhaustive search with the purpose of estimating the vicinity of the solution found by the heuristic with regard to the exact one found by the exhaustive search.
However, from the point of view of the general effectiveness, understood as the speed required to find solutions of similar precision, would be necessary to compare your heuristic with another one previously considered as appropriate to the task that is being solved.
As I understand you are working on GA, so I think you must compare your results with other evolutive algorithms. I am attaching two works where you can find new evolutive algorithms and some its applications
Thank you very much for all your references. Those have been very useful.
The generated hybrids EAs have been compared with the state of the art. However, I just needed this extra help to compare the generation process.
Thank you again.
- Badges
- Science topic
- Similar topics
- Linguistics
- Philology
- Classical Philology
- Manuscripts