It is very time consuming for solving large-scale NP-hard problems. Distributed algorithms based on Map-reduce can speed up the computation and have many successful applications. However, meta-heuristic algorithms such as Tabu search and simulated annealing algorithm are based on single-solution iteration, Hadoop is not good at iteration computation, so few work can be found on distributed iteration meta-heuristic algorithms based on Hadoop or Spark. There is any good implement or idea on iteration meta-heuristics based on Hadoop or Spark? What is its challenge?Thanks.
you can use twister is a MapReduce implementation for the iterative algorithms
I would also suggest Twister. You can also have a look on Storm and Flink.
Best,
Sabeur
You could try to use an generative hyper-heuristic methods using a graph based GP. It could find some algorithms that are human readable as well as robust, in my experience.
We are working on a framework for distributed metaheuristics over Spark. Il will be released around January/February 2016. If you are willing to collaborate you are more than welcome. :)
Take a look at our framework, HyperSpark:
https://github.com/deib-polimi/hyperspark
you can check this:
Tauer, G., & Nagi, R. (2013). A map-reduce lagrangian heuristic for multidimensional assignment problems with decomposable costs. Parallel Computing, 39(11), 653-668.
Chicago
- Badges
- Science topic