Metaheuristic algorithms would be very vital in solving problems related to very complicated inter-relation among control variables such as deep learning and modern communication-related optimization problems. We can observe that there are several research papers in which even simple problems (convex in nature) have been solved using metaheuristics, which obviously not a contribution at all. Actually, researchers should first justify why they are thinking to apply the heuristic algorithms for solving a particular problem. I only think to apply heuristic when the problem will not be solvable using analytical methods. Nowadays, we have several freely available solvers and therefore it is good to first try analytical methods. If it fails to solve the problem only then we should think of applying any heuristic algorithm. Heuristics are the only option when an optimization problem is not solvable using analytical methods.

The above input is based on my own experience as I have worked with both analytical and metaheuristic methods.

Dear sajad,

Metaheuristic optimization techniques now are used in very important applications. However, the major drawback in these methods is the real time application of it. It consumes alot of time. This delay can negatively effect in the system.

Therefor, it will be better if the developers are taking on the account these drowbacks.

Mohammed Bilal Danoune Thank you for your expert opinion.

Great question!

I recommend you this paper, which has been recently published in SWEVO. As you can see, within the coauthors we can find some of the most important researchers of the related community: Article Bio-inspired computation: Where we stand and what's next

Eneko Osaba Thank you professor for the reply. Without any doubt, the article you suggested is really good. The literature survey done is amazing and full of quality content. Moreover, there are definitely active researchers particulary Ponnuthurai N. Suganthan of the Evolutionary Computation (EC) field as co-authors of the suggested paper.

exemple.docx

Metaheuristic algorithms would be very vital in solving problems related to very complicated inter-relation among control variables such as deep learning and modern communication-related optimization problems. We can observe that there are several research papers in which even simple problems (convex in nature) have been solved using metaheuristics, which obviously not a contribution at all. Actually, researchers should first justify why they are thinking to apply the heuristic algorithms for solving a particular problem. I only think to apply heuristic when the problem will not be solvable using analytical methods. Nowadays, we have several freely available solvers and therefore it is good to first try analytical methods. If it fails to solve the problem only then we should think of applying any heuristic algorithm. Heuristics are the only option when an optimization problem is not solvable using analytical methods.

The above input is based on my own experience as I have worked with both analytical and metaheuristic methods.

I think the response from Mahamad is a good one - do NOT use a metaheuristic until you have failed with the most obvious alternatives from mathematical optimization.

I agree with Mahamad and Michael. One should first use exact methods, and then move to heuristics if the problem cannot be solved exactly because of its size or complexity. I guess many people go straight to heuristics because they are easier to implement and usually no modelling skills are required.

In recent years, the field of combinatorial optimization has witnessed a true tsunami of “novel” metaheuristic methods, most of them based on a metaphor of some natural or man-made process. The behavior of virtually any species of insects, the flow of water, musicians playing together – it seems that no idea is too far-fetched to serve as an inspiration to launch yet another metaheuristic.

source: Metaheuristics—the metaphor exposed

The aforementioned replies by researchers like Essam H Houssein and Mahamad Nabab Alam are full of insight. I want other researchers to also share their valuable feedback and raise interesting queries regarding meta-heuristics.

Meta-heuristic algorithms are very suitable and powerful tools for solving complex problems. However, the most important thing to keep in mind is that the performance of many algorithms that have even been published in reputable journals is very poor. On the contrary, some algorithms (like WOA) have great performance over a wide range of issues (structural or high dimensional complex problems even with more than 10.000 dimensions!). In my opinion, it seems necessary to create a database for verifying the accuracy of the methods (their computer codes) that are presented today. In this way, an exact hyper-heuristic process can be created, in which the content of the problem is examined and then the appropriate method to solve it is selected. However, the reason for providing new meta-heuristics methods can be evaluated from two perspectives. The first viewpoint is to develop and improve existing methods in problem solving, and the second one is satisfaction of the scientific level required by universities by providing an article!

I too agree with Farid Miarnaeimi .

Thanks bro ;)

I think this kind of algorithm help scientist to get rid of heavy calculation in complex problems.

I agree with Mahamad Nabab Alam

The no free lunch theorem is not an excuse to implement just any metaheuristic - as none of them guarantee to find even a local minimum. What it does mean is that one needs to search elsewhere for specialised numerical procedures, or devise one that is tailored to your current problem.

I think that that will be a much better strategy, especially if you can utilise some elements from traditional mathematical optimisation tools, but perhaps adjusted a little in order to be slightly more flexible or not as computationally hard, such as incorporating local searches, tree searches for combinatorial problems where there is an upper bound on the number of branches allowed (for complexity's sake) or Lagrangian decomposition tools where we do not necessarily try to close the duality gap.

As the field is so complicated, and constraints may be hard to even be satisfied easily, we will need to combine some of the traditional methods from mathematical optimisation with sensible heuristics that can be tailored to the problem. They will not be all-purpose ones, I'm quite sure.

I think that the perfect knowldge of the problems helps to solve them well. Because sometimes، we have the problem but it needs other basis in several specialities as maths for exact methods for exanple or biology knowldge for using metaheuristics.

Pethaps, we can hybride the two approches to obtain good solutions.

Explain "perfect knowledge", Mohamed.

Because sometimes a problem can be resolved using an exact method taking into acount other constraints linked to the problem that reduce fine the problem complexity . I said this, because I tried with a Np-deficult problem and I arrived to reduce and resolve it. Thanks.

Aha! We mathematicians would call them "valid inequalities." :-)

But that is not always possible, of course, as standard solvers devised - say - in the naughties may not be equipped with up-to-date tools like branch-and-cut. The software in this business is still being developed, and I predict that most of the not up-to-par tools like every meta-heuristic ever coded, will - within a few years - be thrown away, thanks to better numerics and improved theory on combinatorial optimisation. You read it first here! :-)

Good question ... worth pursuing.

I find it interesting - or, rather amusing - to see the posts on this thread, especially when someone writes that they "solve" the problem. Everyone should already know that metaheuristics almost never solve the problem - they may find a better solution than the starting point, but more cannot be guaranteed! And you wouldn't even know that you are standing at an optimum even when you are!

So here is an idea:

And I am still puzzled why people - after having run the metaheuristic - do not utilise that output as a starting point for mathematical optimisation tools that do guarantee to find an optimum.

It would be a rather nice set-up to combine the two, in fact, as first the heuristic might identify a few interesting areas to search in, and in those areas one could use mathematical optimisation approaches, that will guarantee a global optimum over those areas. Together the two strategies may cover more ground, based on a solid strategy.

That could be a nice strategy for some hard problems, I think. They might not yield an optimum, but quite likely something better than the meta method alone.

Michael Patriksson Assuming that the problem is nonconvex, I think combining a metaheuristic with an optimization method cannot guarantee getting closer to a global optimum than when more iterations of the metaheuristic alone are performed.

I am not talking about a combination of a meta-thing with a global solver, but a stand-alone global solver. As long as the problem is defined by Lipschitz continuous function, I think that the global solver will identify an optimal solution more often than a meta-thing. But it would be fun to have a battle. :-)

Metaheuristics are powerful methods for simulation-based optimization.. several metaheuristics techniques such GA, Tabu Search, etc. are used to capture real-life optimization problems.. normally known as non-deterministic polynomial (NP) hard problem or complex combinatorial optimization problem. The classical optimization methods become inferior or ineffective for solving these kinds of optimization problems.

Consider the TSP, which is NP-hard. For this problem we now have exact solvers that find optima every time, and fast. The thing that us mathematicians do is the only way to yield optima - and optima is the only solution we accept, as we will use that output for actual designs in industry.

A metaheuristic, as you know, is not even designed to find an optimum, and therefore it very seldom terminates with an optimal solution.

If you are ok with the output from a metaheuristic, then that's fine, and I am glad that you find them useful.

But for us, it is generally not acceptable.

one of the fastest and high-quality papers is slime mould algorithm SMA, it has a high potential for publication in best level journals:

Article Slime mould algorithm: A new method for stochastic optimization

Code Matlab code of Slime Mould Algorithm (SMA)

Follow

Conference Paper A Two-Stage Matheuristic Algorithm for Classical Inventory R...

Article Deep Reinforcement Learning for Solving the Vehicle Routing Problem

that's right, every year new optimization algorithms are being developed by researchers and then, they start solving same kind of problems previously solved by their classical counter parts . this problems is bench mark of the most popular test problems in structural optimization, which has already been solved in several articles and other widely used metaheuristic methods are relatively well known not only among computer scientists but also scientists active in other fields. your question's and The question that is asked by the judges of articles and philosophers in the science of optimization is how the superiority of the newly introduced algorithm compared to other meta-heuristic methods can be compared and evaluated. The answer to this question can be provided by optimizing the weight of structures and comparing the results using parametric and non-parametric statistical tests.

Metaheuristic algorithms can be summarized as a form of stochastic optimization algorithm which does not depend on the surface gradient for optimization. These algorithms draw inspiration from various sources to improve fitness but most of these algorithms are nature inspired. They have been actively researched area due to their vast applications in engineering and artificial Intelligence application. This research area and their applications has been tremendously growing as shown through the last 2 decades research. In this paper a Review on Metaheuristic Algorithms: Recent Trends, Benchmarking and Applications is presented, https://ieeexplore.ieee.org/document/8843624

In the best of worlds metaheuristics would disappear altogether, as we have found the right theory and the right tools to solve all problems that we face.

Unfortunately this will take some time! :-) I fear that the tongue of a deity will not be helpful - only hard work and perhaps a collaborator will do the trick.

In the mean-time I would like to propose to meta-heuristics enthusiasts to try to better combine your approaches with some elements of mathematical optimisation, as those are the ones that more often will succeed, thanks to the stronger ties to the theory behind mathematical optimisation and in particular the theory of optimality conditions - which is what we aim to comply with. Fruitful collaborations over the science borders might be a very good thing - at least for several of the problems we face today.

Metaheuristics are problem-independent techniques that can be applied to a broad range of problems.

The "curse of metaphors" has plagued the meta-heuristic domain for a long [1] and [2]. The need of an hour is to rigorously investigate the underlying computational processes (by stripping away the trivia) and benchmark them against the state-of-the-art DL approaches. The 'publish or perish' dynamics of the academic world are also not being helpful in this regard. Lately, the meta-heuristic domain has become a past time vocation with limited practical relevance, capitulating more ground to the competing frameworks.

[1]. Villalón, C.L.C., Stützle, T. and Dorigo, M., 2020, October. Grey Wolf, Firefly and Bat Algorithms: Three Widespread Algorithms that Do Not Contain Any Novelty. In International Conference on Swarm Intelligence (pp. 121-133). Springer, Cham.

[2]. Camacho-Villalón, C.L., Dorigo, M. and Stützle, T., 2019. The intelligent water drops algorithm: why it cannot be considered a novel algorithm. Swarm Intelligence, 13(3-4), pp.173-192.

I think hybridization of meta-heurstic algorithms are not much used for solving the optimization problems specifically 3 or more hybrid algorithms. Before making a step toward Article Intelligence, we have to think about hybridization of algorithms.

I am rather bored by the extensive use - or should I say mis-use - of metaheuristics, as they are really unreliable. Those who use them - do learn how to model your mathematical optimisation problem by well-chosen variables and constraints, and use mathematical optimisation tools in order to solve them instead. By all means let a metaheuristic find an initial feasible solution, if it is not too nasty. But let the world of mathematical optimisation solve the problem, please.

I am agreeing that developing new metaheuristics are not useful and they look already the same. The so-called novel metaheuristics are not interesting, even for me as an enthusiast. I guess that the hybridization of available heuristics with mathematical programming would be a better choice. I have started this discussion at the following link:

https://www.researchgate.net/post/What_do_you_think_about_improving_metaheuristics_using_mathematical_programming

To be a metaheuristic enthusiast is to be less knowing. When you apply a metaheuristic you already know that it will be inferior to a math programming-based methodology. I don't know why one would pick the inferior methodology, but I have long ago stopped being surprised.

I am interested in both mathematical programming and metaheuristics. As Dr. Michael Patriksson mentioned, mathematical programming is superior and more reliable. I will choose mathematical optimization as the default tool (even for my new papers, I am using a gradient-based algorithms). However, for cases in which the calculation of analytical sensitivities is impossible (which happens in many engineering cases, specially in structural optimization), heuristic optimization is the only available tool. Prof. Michael Patriksson , you cannot claim it as a lack of knowledge. If you don’t like something, it doesn’t mean to be useless!

My recommendation to this question is that combing the available heuristics with mathematical optimization is much better than creating a so-called novel heuristic.

Sorry for being blunt, Pooya Rostami but for implicit functions I would utilise a model-based approach such that what I optimise is an explicit function based on the samples of the function that I have devised. I have done such papers with industry, and some of them I think are still used in their enterprise.

Dear Professor Michael Patriksson :

I am not sure that I have understood as well, but I guess that you are referencing the surrogate models. In my case study which is highly nonlinear and high cost (with some possible discontinuities) and large scale, the surrogate modelling will not work. I have tried Kriging models and machine learning which failed for inefficiency. I think that surrogate modeling will not work as well in all problems.

Regarding your industrial projects, I know that you have been involved in problems such as tire optimization which is a very nonlinear and complex problem due to hyperelastic behavior and air resistance or automotive safety problems which is my interest. Have you solved them all using mathematical programming!?? Since the determination of gradient information is very difficult in such cases! In how dimensions a model based method worked?

Dear Professor Michael Patriksson :

To complete my previous reply, regarding the use of surrogate modelling. In the following link regarding the question “Why surrogate optimization is best suited for expensive (time-consuming) objective functions?” you have commented: “That claim is not always true ….”:

https://www.researchgate.net/post/Why_surrogate_optimization_is_best_suited_for_expensive_time-consuming_objective_functions

So are you agreeing with me? If yes, the heuristics would be the only available tool for some problems with lack of knowledge as mentioned by IBRAHIM GÜNAL

Radial basis function interpolation has been one of the tools that we have used with VOLVO. It does help quite a lot for problems when we can only calculate merit functions through simulations. We have had much use from those tools. And we can utilise math programming tools, which kills any metaheuristic. The verdict is indeed out.

Indeed we have been using interpolating functions in order to solve some of the toughest problems (RBF interpolation) - and it works!

Dr. Michael Patriksson :

Bravo! Very nice implementation! I will try it for my case. Thanks a lot for the reply.

Pooya Rostami Let me know if it works well for you!

I think that in the future, metaheuristic algorithms will be integrated with artificial intelligence as part of a strategy for finding the optimal solution.

But I got agreed with Michael Patriksson, specially in continuous structural optimization which is my case.

Generally, in the cases of with the existence of sensitivity, mathematical programming is preferred and works on very large scales efficiently. In the cases with the lack of knowledge and unavailability of gradients, we can use a model-based method to approximate objective function or predict the sensitivity coupled with mathematical optimization. So, we no longer need metaheuristics and they can be disappeared.

I have started this discussion at the following link:

https://www.researchgate.net/post/The_improvement_of_model-based_methods_and_surrogate_modeling_kills_metaheuristics_in_continuous_optimization_What_do_you_think