Let us start with a simple 1+lambda-ES. The possible search points will be centered around the current position. I'm going to call this centering exploitation. In general, given a point, we can find a local optimum near that point. So, the real challenge is (for multi-modal search spaces) to generate interesting (non-random) search points away from the current position.
In PSO, particles can have search trajectories influenced by multiple neighbours (i.e. lbests) and momentum. Although a large number of the new positions will be close to one of the existing positions, there is a useful opportunity for search among the various attractors.
Opposition-based learning will create search points at the opposite side of the search space. ABC generates random scouts that then do an almost "path-relinking" search towards an attractor. DE uses difference vectors.
What are other interesting exploratory methods? (Note: if possible, please also explain the rationale/expected benefit, and not just the means of generation.)
A good question Stephen.
I just wrote a short paper on this topic which you might find relevant. It's about metaheuristics (in the sense of search strategies) used by nature-inspired algorithms, but also touches upon other approaches:
Metaheuristics in Nature-Inspired Algorithms
http://www.macs.hw.ac.uk/~ml355/common/papers/lones-gecco2014-metaheuristics.pdf
Most of these algorithms are some combination of 'centering exploitation' (or neighbourhood search) and path relinking (or intermediate search). Other interesting strategies include iterated neighbourhood search, which estimates the distance between optima in order to sample new points, and variable neighbourhood search, which involves exploring neighbourhoods of different sizes in order to avoid local optima.
Yes, a useful categorization of metaheuristics. In general, everything works great for globally convex search spaces -- local search and intermediate search all work best when good local optima are clustered.
Since I will also claim that random search should be used in random search spaces (e.g. the Gallagher functions in BBOB2009), is there any point to worry about non-convex search spaces? (i.e. just use random)
ps - I love the Sorensen paper.
https://www.researchgate.net/publication/297245624_Particle_Swarm_Optimization_Algorithm_and_its_Codes_in_MATLAB
Research Particle Swarm Optimization: Algorithm and its Codes in MATLAB
https://www.researchgate.net/publication/296636431_Codes_in_MATLAB_for_Particle_Swarm_Optimization
Code Codes in MATLAB for Particle Swarm Optimization
https://www.researchgate.net/publication/280244004_A_comparative_study_of_metaheuristic_optimization_approaches_for_directional_overcurrent_relays_coordination
Article A comparative study of metaheuristic optimization approaches...
- Badges
- Science topic
- Similar topics
- Engineering
- Modeling