I have heard that a good way of testing a metaheuristic is by using random generated problems, but I am not sure if these can be considered as standard test functions with random parameters. If so, could anyone please share some references (and/or examples) on this topic?
If there are no other benchmarks, then random instances often became the first benchmarks. However, for most popular problems, there are now suites of standard instances that are useful for comparisons. See for example:
http://coco.gforge.inria.fr/doku.php?id=bbob-2012
http://www.ntu.edu.sg/home/EPNSugan/index_files/CEC2013/CEC2013.htm
http://comopt.ifi.uni-heidelberg.de/software/TSPLIB95/
Thanks for your quick reply. I already checked them and I think I can make very good use of the information. Thanks again...
Just adding some more references (Stephen already gave some good starting pointers): SteinLib, MipLib. The OrLib was maintained by J.E. Beasley. You can still get it at http://people.brunel.ac.uk/~mastjjb/jeb/info.html Instances from many problems are available there. ZIB from Berlin hosts some libraries too (http://www.zib.de/en/optimization/software.html). Aside from those references, many libraries are available all over the world. It just depends on the problem you are working on.
Now, is it ok or not to use randomly generated instances? For problems strongly connected to real-life situations, it is far better to work with real data, otherwise the most realistic possible data. This raises some questions on the bias and properties of the instances: do graphs have to be planar, acyclic? Should the costs be unform in (1;5), and so on...
Thank you for your comments Duhamel. I want to test a modified version of an algorithm against the original one, so I am thinking of using monomodal and multimodal objective functions, in different dimensions (e.g. Rosenbrock, Rastrigin, Schwefel, etc...). I heard about these random generated problems, and after checking some of the links, I think it would be a nice addition to the testing scenarios.
- Badges
- Science topic
- Similar topics
- Engineering
- Modeling