When comparing two optimization methods on a function, should we use a two sample t-test or a paired t-test? I would say we should use the latter since paired t-test is used for correlated observations and in our case we can consider the unit of observation to be the function and the two methods as two treatments. Am I right?
Thank you in advance
these tests is suitable for nearly obtained solutions in comparing EAs, I suggest you that employ RDI and RPD and Average best solution, and ... to compare EAs.
see this : https://www.researchgate.net/publication/240372025_Optimizing_a_Multi-Vendor_Multi-Retailer_Vendor_Managed_Inventory_Problem_Two_Tuned_Meta-Heuristic_Algorithms?ev=prf_pub
Article Optimizing a multi-vendor multi-retailer vendor managed inve...
A novel statistical approach for comparing meta-heuristic stochastic optimization algorithms according to the distribution of the solutions in the search space is introduced, known as extended Deep Statistical Comparison. This approach is an extension of the recently proposed Deep Statistical Comparison approach used for comparing meta-heuristic stochastic optimization algorithms according to the solutions values. Its main contribution is that the algorithms are compared not only according to obtained solutions values, but also according to the distribution of the obtained solutions in the search space. The information it provides can additionally help to identify exploitation and exploration powers of the compared algorithms. This is important when dealing with a multimodal search space, where there are a lot of local optima with similar values. The benchmark results show that our proposed approach gives promising results and can be used for a statistical comparison of meta-heuristic stochastic optimization algorithms according to solutions values and their distribution in the search space. For more information, you can refer the following paper.
- Badges
- Science topic
- Similar topics
- Mathematics
- Statistics