The response from those who know about both - such as me - is that PSO and other meta things will very unlikely be as robust and accurate as a modern branch-and-bound code. I have always been puzzled why people seem to not be interested in what the optimum actually is - I thought that finding an optimum was the whole idea with using optimisation in the first place!

And yet there are more people here that seem to be content with non-optimal points. I would not be, and therefore I do not use meta things, and instead use tools that guarantee that the final output always is optimal. I have always thought that that is the whole purpose with the use of optimisation tools. But for some I guess not. I can't understand why - it seems to be a waste of time.

If you use them at your workplace, you will have to show your boss the results of your computations; aren't you afraid to confess that you have used bad methods - or are you going to keep the secret to yourself? That might lead to headaches for everyone at the workplace, ...

The answer hence is: You should not use any of them, if you do not have to, as almost none will lead to an optimal result - and you wouldn't know anyway - and you do not know how much you lose by using these inferior methods. I find it terribly sad, and ultimately extremely silly. The "No free lunch theorem" is not a scam, but its utilisation is exaggerated to the extreme: very few combinatorial/integer programs are really hard, and there is very little risk that yours is one of them.

You are digging a hole in which you will fall into, and you won't be able to climb up, and you even do not want to climb up, since you are now familiar with these inferior tools - a metafor meaning that if you "believe" that meta things are superior then you won't listen to good advise. That is the situation: you have to be content with inferior tools, as you are not knowledgeable with the better tools, and rather than admitting it you simply soldier on, believing - in a sort of religious fashion, even - based on faith rather than on knowledge.

And that last bit is the key: you have to use them, because you do not know anything else.

The key to empowering yourself is to learn mathematical optimisation tools. That is the only way to ultimately gain anything from your attempts to solve whatever problems you face.

I don't have much experience of working with PSO and I never tried ABC. However, I can tell you a fair thing or two about GA: it's fast, easy and guarantees good solutions (can't say the same for optimality), especially for discrete problems. Also, there's tons of literature available. So, I think you can try it once.

I have worked with PSO. Its easy, fast and has less parameters and computations. On the other hand, it is a little tedious for multi-objective problems, when the solution sets are discontinuous and scattered and easily converges to local minima/maxima. GA perfoms well compared to PSO for multi-objective problems. Basically, the choice of optimization algorithm depends largly on the scope of the optimization problem.

The response from those who know about both - such as me - is that PSO and other meta things will very unlikely be as robust and accurate as a modern branch-and-bound code. I have always been puzzled why people seem to not be interested in what the optimum actually is - I thought that finding an optimum was the whole idea with using optimisation in the first place!

And yet there are more people here that seem to be content with non-optimal points. I would not be, and therefore I do not use meta things, and instead use tools that guarantee that the final output always is optimal. I have always thought that that is the whole purpose with the use of optimisation tools. But for some I guess not. I can't understand why - it seems to be a waste of time.

If you use them at your workplace, you will have to show your boss the results of your computations; aren't you afraid to confess that you have used bad methods - or are you going to keep the secret to yourself? That might lead to headaches for everyone at the workplace, ...

The answer hence is: You should not use any of them, if you do not have to, as almost none will lead to an optimal result - and you wouldn't know anyway - and you do not know how much you lose by using these inferior methods. I find it terribly sad, and ultimately extremely silly. The "No free lunch theorem" is not a scam, but its utilisation is exaggerated to the extreme: very few combinatorial/integer programs are really hard, and there is very little risk that yours is one of them.

You are digging a hole in which you will fall into, and you won't be able to climb up, and you even do not want to climb up, since you are now familiar with these inferior tools - a metafor meaning that if you "believe" that meta things are superior then you won't listen to good advise. That is the situation: you have to be content with inferior tools, as you are not knowledgeable with the better tools, and rather than admitting it you simply soldier on, believing - in a sort of religious fashion, even - based on faith rather than on knowledge.

And that last bit is the key: you have to use them, because you do not know anything else.

The key to empowering yourself is to learn mathematical optimisation tools. That is the only way to ultimately gain anything from your attempts to solve whatever problems you face.

If it is a 1-objective linear or quadratic problem, then the modern optimizers can solve it quite quickly (cplex, gurobi, etc.). If it is 2-objective, usually it's the same thing using the epsilon-constraint method.

In other cases, or if you really want to use a metaheuristic, any of those PSO, ABC, GA is simple enough to implement and run. But be prepared to deal with different results every time you try it :)

PSO will be elegant and best suited

It feels like you didn't try to use an exact method. Thus, I have to write here that all the beginers are suppose to listen to prof. Patriksson. So, at first try to find an optimal solution. Use CPLEX, learn to model. This will be one of your first sentences in the research paper: "We tried to solve this NP-hard problem... (provide some proof that you really tried)... Because we are not able to generate an optimal solution in reasonable amount of time we decided for a heuristic approach..." I hope this is clear. Otherwise, it is like working with machine learning without knowing about statistical tests. It is slightly embarrassing.

Thanks, Tatjana! :-)

I also do think it is quite embarrassing that those who happily use metaheuristics appear to NEVER EVEN ONCE try to use a branch-and-bound code, like CPLEX, as referred to by Tatjana above. Do it once, please, and if you do not think it works for you, then by all means keep on using your favourite tools. But at least try ONCE - and compare!

PSO will be elegant for complex problems and large scale problems.

Dear Jitendra Singh, I want to answer the part of your question that I didn't address before. As a beginner you want to know, to a certain detail, about all three methods: PSO, ABC and GA. If you work in the research area, you will most probably have to compete with one of this methods at some point. You will have an opportunity to decide which one you prefer and continue to use that one. Beacuse, No Free Lunch Theorem says that you can't go much wrong in any case.

E. Moosavi Elegant? No method should be chosen because it is elegant - what utter non-sense!

A method should be chosen because it is favourable with respect to the problem that it solves, and also (sometimes) to the computer machinery that one has at ones disposal.

I could list very many methods that are elegant in a way - but damn useless just the same. Some of them happen to be metaheuristics.

Tatjana Jakšić Krüger The NFL theorem is not useful very often anymore, as software gets better and better, and cycle times on the computer are shrinking. There is, I would say, less than 1% of the problems that now are computed (erroneously) by metaheuristics will still be attacked by meta stuff in a few years.

The other 99% will have the help of a preprocessor that analyses what type of problem it is, how large it is, how degenerate it is, what nonlinearities there are, and so on. From that a specialised tool will be built that take into account the original data, and construct the right methodology. And it will provide a near-optimal solution quickly.

I once worked on a large linear problem which a GA solved very quickly, but cplex solved exactly even faster. Just a split-second, actually.

prof. Michael Patriksson I am still a beginer, however, long enough in the field of OR to recognize that beside failings of MH (metaheuristics) for lack of standardization, they are easier to implement than most of the exact methods. One does not have to be concernd with mathematical side of the problem, modeling, or even fully understanding the problem. The same things can be considered as deficiencies, and we covered that at the start of this thread. However, as long AI algorithms continue to develp in order to fix everything with a hammer, MH are not going anywhere.

In our bioprocess-based three optimizations, PSO outperforms the GA and ABC. Its better to go with PSO in my opinion!

The position to take is to not use any of the tools PSO, GA, or ABC - PSO is not reliable at all. They are terribly unreliable, and they almost always never deliver an output that is close to being optimal. I find it hard to fathom the stubbornness, when there are very good implementations of methods that WILL give you optima, and you do not need to check for errors, because the tools are practically free from bugs - they do deliver an optimum, or a near-optimum if you opt for an output based on a CPU-time limit.

Compare that to buying a car with three wheels, rather than four. That's what most metaheuristics are, and how bad it can get, if you catch my attempt to illustrate.

And to have an opinion on something that is not tested should be avoided.

Vijay: your conclusion only applies to your narrow topic - not for the whole spectrum of general problems. It's very important to not hint that a method that works for one problem will be good for any other ones - my experience in mathematical optimisation during 30+ years is very solid when I say that just because it works for a few sample problems, it may not be for a larger test set of problems.

Tatjana Jakšić Krüger The "only" bug then with meta stuff is that none of the methods in that class provides an optimum solution.

I find that bug essential. Therefore I do not use them.

Michael: sure, if exact methods exist which solve a particular problem in reasonable time, by all means they should be preferred.

Just that sometimes you don't have such luxury, in which case you need to use other methods. If an approximation is good enough, then metaheurisitics may be helpful and this is why they should be researched.

And, as you certainly know, they also give rise to interesting mathematical problems per se. An important class is proving whether or not a particular metaheuristic converges to the absolute optimum. These proofs are usually very difficult and challenging.

@Michael when you consider the computer power speed improvement on helping non meta approaches to solve optimization problems. Do you also consider in this phrase how heavier these problems have become, for example many-objective problems or problems with too many decision variables such as Deep Learning optimization and generation?

The must important aspet of any problem is it adequation of the problem to be solved. For finding a good dscription you must to do firstly a system analysis. On this basis you could find the adequate conceptual mathematical problem to be solved. in the case of obtainning a Mixed Integer programming model, using or not heuristics methods depends on the complexity of your problem. In the case of multicriteria multilevel model it is difficult to avoid using heuristics. In my profile you can find some ways of solving such king of problems.

Escuse me for my English previous redaction.

The must important aspet of any problem is it adequation of the model to th problem to be solved. For finding a good dscription you must to do firstly a system analysis. On this basis you could find the adequate conceptual mathematical model to be solved. in the case of obtainning a Mixed Integer programming model, using or not heuristics methods depends on the complexity of your problem. In the case of multicriteria multilevel model it is difficult to avoid using heuristics. In my profile you can find some ways of solving such kind of problems. in my reseachgate profile you can find systems analysis procedures and new heuristiss that could be useful

For all of you above - a short announcement:

First I explore several variable and constraint declarations that will yield the same optimal value. (Some tend to be classical, but one can manipulate the model such that the optimal value is the same, by using a variety of (variable/constraint) declarations. This should nowadays be standard, as there are typically many ways to declare the model through variable and constraint variations.

THEN I run tests on realistic instances, and draw conclusions on speed, and some times also on memory requirements. This is typically done using sophisticated branch-and-bound methods that are robust, and never fails to yield an optimum solution.

I rarely use meta stuff, as they do not provide optima, but I have at times with serious scientists had battles between different metaheuristics, as well with optimal solutions. Sometimes the meta solutions are not that bad in comparison, but they typically do not come even close to optima. I do not understand why meta people rarely perform comparisons, including a comparison to optima - at least I have not seen one. THAT is odd, and even slightly suspicious.

I would like to have feedback on this, as we really should collaborate across scientific borders!

I tend to NOT consider multicriteria problems these days - although I have done several such projects. The industrial problems I have had with, for example, VOLVO or aircraft companies have had one objective function, and there are already enough complications ... :-) I am more interested in solving problems efficiently, which means that I test several equivalent models (i.e., modelling with a variety of definitions of the variables and/or constraints) in order to strengthen the models' 1/0 properties - for a more efficient solution using branch-and-bound techniques. I find that more rewarding - and also that is a more solid math-based technique that always provide an optimum. I DO NOT accept a non-optimal solution.

well... weighted-sum works fine for multicriteria. e-constraint too, but for 3+ objectives the solvers tend to get stuck (at least for the problems I deal with)

Recently I Have seen several real-world many optimization problem, for example the generic aircraft design which consists of 10 objective functions. Because of that I tend to agree with Nuno Miguel Marques de Sousa and José Arzola-Ruiz However, If we could always do as Michael Patriksson does It would be the best possible option.

There are several scalarizations for multi-objective optimization problems, see e.g. the article by J. Jahn

Chapter Scalarization in Multi Objective Optimization

From my point of view having to deal with a multi-objective problem means that in real-world problem there are several aspects to be considered and the "real decision maker" is unable to specify just one objective or fixed weights for the objectives given. Every Pareto-optimal point is a candidate for a "best compromise". Still in practice only one solution could be implemented (like in a production plan or the construction of a structure like the mentioned airplane).

From the theoretical point of view it seems to be the aim to determine the whole Pareto set, but in practice this is not the real goal - and not just because of the possibly enormous amount of Pareto-optimal points.

I think that besides the specified objectives there are other aspects of the solution points leading to preferring of one of these candidates oven another which the decision maker is unable to incorporate in the model directly.

From my point of view an interactive procedure is the best way to solve such problems: produce Pareto-optimal points (which could well be done by using one of the scalarization techniques and apply "normal" optimization software) and present the solutions to the decision maker. Based on these he should experiment with the control parameters of the scalarization (change weigths and/or aspiration levels), resolve and evaluate the changes in objectives and variable values until he is satisfied.

I worked in the past with the reference point approach of A. Wierzbicki (minimizing the weighted Tchebycheff norm distance to a reference point in objective space, a generalization of the displaced ideal approach), since the control is better than with just weighted sum, for multi-objective LPs it can produce non-vertex solutions, too - and it works for nonconvex settings, too.

This could well be done without resorting to heuristics.

by Seng Poh Lim & Habibollah Haron in 2013 have been compared performance of three metaheuristic algorithms.

DOI: 10.1109/ICOS.2013.6735045

see this book:

Talbi, E G, 2009. Metaheuristics: from Design to Implementation. John Wiley & Sons, Hoboken, New Jersey, USA.

The industrial cases that I have been involved with have mostly had one (1) objective. They decide - and I can't judge what they want. However I can tell them that some combinations of variable declarations and constraints yield perhaps several - and equivalent - problems, that one can have competitions with, i.e., models that will yield the same output but perhaps - depending on the solvers that are used - will be easier to solve. Real industry tend to have one objective (but not necessarily a simple one), while academic ones can have several. I have more than a hunch that one objective is still the norm - and I have seen plenty.

Alireza Ghasemi Have you had a competition among your tools and some mathematical optimisation tools as well? That might be enlightening, especially due to the fact that metaheuristics almost never find an optimal solution - and frankly - mostly misses totally.

Given that mathematical optimisation tend to virtually kill metaheuristics I do not for the World fathom that metaheuristics are so popular! Can anyone tell me WHY??

I cannot understand why you do not use the tools that always bring you an optimum solution (hint: mathematical optimisation), and instead - against common sense!!! - drag a metaheuristic through the mud, with the obvious result that you will not receive an optimal solution - by very far.

If you then implement it, you may get a very angry phone call from the company that hired you, since they have - unbeknownst to you - run a branch-and-cut solver on the web for free, showing that the optimum is actually more than 15% less expensive than your code's output. It's actually insane.

Saad: here is a tutorial page, with links to solvers:

https://realpython.com/linear-programming-python/

Saad: let me know if you run into problems.

I would always try an exact solver first. There are some excellent ones out there: CPLEX, Gurobi, Xpress, SCIP...

Perhaps we must point to where it can be found. But a google search on terms like those will very likely lead directly to the right page. If for some reason you fail, alert me and we will help.

Michael, as you know, metaheuristics are based on simple, yet ingenious, nature-inspired mechanisms. They carry an aura of romanticism which can be alluring. And, of course, may be best choice for some problems.

Nuno Miguel Marques de Sousa - But they are not based on any notion of optimality, which - on the contrary - mathematical optimisation codes are. They are the ones to seek, and apply! Ingenious is not the right word for metaheuristics at all, but perhaps "handy," or "simple." For me metaheuristics go against my wish to find an optimal solution, and I have never ever seen a metaheuristic getting it right. Have you? How can one state that it is a good methodology, when they almost never find an optimum solution?

And because you do not know about the theory of optimality, you will not know what your code gave you as output - is it even feasible?? I urge you to use common sense!

Realise that - while metaheuristics are simple to apply - they are not based on a solid foundation. Therefore it almost always fail to find an optimum - and since the actual optimum very often is much, much better than the metaheuristic will find, you also risk losing so much in terms in the optimal objective value of the problem that you will regret it - and you might also get into real trouble when you have to explain to the boss that you did not try hard enough.

Mathematical optimisation tools are built upon a very solid theory of optimality, and therefore they do not often fail - and we also know very well when, and why, some problem is hard to find the optimal solution to. We're that good - or, rather, the theory is that solid.

I doubt it. Bosses will be content with 80% of the solution in 20% of the time. And metaheuristics can do that :)

I find it very hard to believe that you can back up your claim, Nuno Miguel Marques de Sousa - and YOU cannot know, unless you actually solve the problem optimally, to make sure! You have no idea how bad the output can be from a metaheuristic. You're very wrong.

I'm confused by some of the answers here (some are fine, others just are strange); I think Michael is on point as usual .I always defer to the more basic question: What is the MILP? I can't answer the question without knowing what the problem is, unless you truly want to just solve MILPs more broadly or just "want something" to solve some MILP without much more deeper consideration of its formulation.

There might be a much (much) more efficient and simpler algorithm for your MILP than you might realize, if you actually sit down and study the problem formulation. Going to things like meta-heuristics is a last-ditch effort when no other things work well or the problem completely lacks structure to do anything with it.

For example, is the MILP for a NP-hard problem? If is not, then why use these techniques (sometimes a formulation looks difficult, but it actually has lots of structure to play with e.g. total unimodularity, or some constraints are redundant)? If it is NP-hard, then have you tried actually drilling down and studying the actual problem? If you are new to programming and other things, why are you jumping on these specific (arguably, less reliable (formally at least), more advanced) techniques and not more standard ones from combinatorial optimization and well-known techniques more broadly from mathematical optimization (e.g. mathematical programming)? Furthermore, there more concrete ways to ensure algorithms yield provably close answers to the optimum (such as approximation algorithms), have you considered these yet?

If you are wanting to not actually study the problem and throw the problem at some black box and that's acceptable enough, why not use a solver? Things like CPLEX or others work quite well for what you might want to do. I understand that might not be "as sexy", but it is indeed a valid way of approaching this and is quite standard to do.

This is not to entirely diminish the techniques mentioned, they've had popularity in the literature empirically (there are exceptions to this rule, where things are done much more formally), however, it would be very unwise to throw this stuff at a problem just because one has not considered the wide chasm of theory that one could explore instead first. This is a valuable reason why learning more Computer Science/Maths (in particular more about algorithms and theory) might be valuable for you, as opposed to just diving headfirst blind. Normally, as an algorithms/theory guy I like being able to prove theorems to guarantee the algorithm is something that actually works the way I claim it does, or if not, at least I have some formal mathematical justification that it will at least do so closely. I don't know if you are holding yourself to the standards of a theoretical computer scientist, but I think me mentioning these things might help guide you.

Very good, Daniel.

I agree with Dr. Daniel Page

A short and obviously a biased answer: Don't use mixed integer linear programming. The following is usually true: if the problem you are solving requires some form of sequential data, and if you impose some realistic limits on execution time or number of iterations . . . . . . the value of the solution that you get is strongly influenced by the ordering of the sequential data, and if you randomly sort that data before running programming you will observe that each ordering give a different solution. With that said genetic algorithms that operate on the sequence of the input data will generally yield good results. There is a corollary that says that the sequence of decisions made within your algorithm will determine the outcome, and a genetic algorithm that operates on the sequence of decisions will yeild good results, and finally if what I say holds true for the problems and methods that you are using, you will find that without using a genetic algorithm but randomizing the input data or randomizing the sequence of decisions will yeild a collection of solutions with a gaussian distribution (there are a few details thatare covered in my publications). Let me know if you would like to discuss.

1. I don't use sequential data, whatever it means. I deal with one problem only, at a time.

2. I am not in a hurry to get an optimum, but an optimum is a must.

3. I will never implement any meta stuff, as it will never will give me what I want: an optimal solution.

Before you think about solvers, there are plenty of things one can do to make sure that the output will be as good as possible:

Perhaps the most important thing to really think about is how to declare the variables. What beginners tend to do is to declare as few variables as one can, while the model is a correct representation of the problem.

This was the standard 30 years ago, when solvers were inadequate, the computer speed was as that of a sloth, or a snail, and the memory size available also was way below today's standard.

In almost all cases I have seen or been involved in, it has been quite favourable to have many variables - the more the better, actually - as it will be easier for the solver to arrange computations more easily. In the 80s that was a very different story - some times one had to stop prematurely because of memory problems. While we still sometimes utilise what we call column generation - in which a subroutine based on parts of the data create a new column (equivalent to a new variable in the problem) - we do it rather less often.

So while you have a problem set up already, by all means discuss if you can use an alternative variable declaration. In fact you might be able to do something similar with the constraints as well.

Good luck!

interesting

I AGREE WITH YOU

I suggest trying to solve the problem to optimality first, before trying evolutionary algorithms. Solve it using CPLEX, and if it is difficult, try to identify special structures in the model. Try decomposition: Benders or Dantzig-Wolfe.