In most MOEA publications, researchers will use IGD or hyper-volume as their evaluation metrics. However, in real-world problems, both nadir points and ideal points are unknown. What's worse, in many cases, the domain knowledge is unavailable. Thus, in this case, how do I compare the performance between different algorithms? For example, how do I compare the performance of different MOEAs on a real-world TSP problem without any domain knowledge?
Dear Hengzhe Zhang,
I suggest you to see links on topic.
https://link.springer.com/article/10.1007/s00521-017-3057-x?shared-article-renderer
https://www.researchgate.net/publication/303449167_BMOBench_Black-Box_Multi-Objective_Optimization_Benchmarking_Platform
https://www.researchgate.net/publication/238184565_An_Efficient_Pareto_Set_Identification_Approach_for_Multi-Objective_Optimization_on_Black-Box_Functions
https://link.springer.com/article/10.1007/s10898-015-0270-y
https://www.sciencedirect.com/science/article/pii/S2352711020300911
https://www.researchgate.net/publication/334385794_Noisy_multiobjective_black-box_optimization_using_bayesian_optimization
https://link.springer.com/article/10.1007/s40747-019-0113-4
Best regards
Dear Hengzhe Zhang,
as Iago Augusto Carvalho said you can approximate the nadir point and use the hypervolume indicator. However, for a fair comparison, one quality indicator is not enough. When I was comparing algorithms on a real-world problem I created my own reference front. I combined the solutions of all the runs of all the algorithms to get a single front. Then I could use all the quality indicators that require a reference front.
Iago Augusto Carvalho sorry for the late response. I saw this approach in some papers. I can't remember all of them, but I found at least one where they used this method:
Article A Multi-Objective and Evolutionary Hyper-Heuristic Applied t...
- Badges
- Science topic
- Similar topics
- Computer Science and Engineering
- Programming Languages