For more than 50 years practitioners have been asking this question; probably the most common.
It is believed that there is an answer, not from the technical point of view, since it is assumed that all methods are mathematically correct, (although I doubt about this.....), and thus, there is no method better than another.
I have said this many times, and it is taken as an axiom by many researchers, albeit for some specific problem it is recognized that some MCDM methods are more appropriate than others
However, can we say that in general a method is better than another?
I think that we can.
This assertion of mine, is based comparing for any problem the operational aspects of each method, on a certain common aspect, easily measured, and it is how well a method can model and solve a problem. It is obvious that if method A is able to model 6 characteristics of a scenario, while method B can model and solve only 4, A is better than B.
Obviously, the result from A will have more reliability that that from B, which is product of only a simplification of reality.
Consequently, it would be interesting to determine a set of special characteristics from many different scenarios, and see how many of them can be modelled by a series of methods.
This is the base for a method that I suggested about two years ago, and that was published in RG under the coding 300a and 300b.
What is your opinion?
I personally prefer value-focused multi-objective decision models, if the decision complexity necessitates this more expensive approach. I think it ultimately comes down to using the right method for the decision situation (which implies using the simplest model, adhering to axiomatic decision analysis, that provides valid results). What I can definitively say is that AHP (and it's derivative methods) is the worst MCDM methodology due to several issues, not the least of which is the rank reversal of alternatives.
As it is known, the best MCDM method is either not available or it is a very difficult puzzle to know.
This problem concerns many interdisciplinary applied sciences. So please let me make a metaphor for common understanding. Let's start with a similar question: What is the best language? (English, French ... et al.) The common impression is that, at first glance, the answer is relative. The purpose of languages is the same. The purpose of MCDM methods is the same. Languages try to model reality with symbols. MCDM does this with numerical symbols.Languages express the same thing with high similarity. MCDM methods also often produce similar rankings. Languages evolve over time to best express a reality. MCDM methods are constantly being developed to find the best alternative. It is difficult to compare languages on a comprehensive and common objective measure of value that they create. It is difficult to compare MCDM methods on the basis of an objective value they create. Languages can express a 'meaning' well or not. MCDM rankings likewise produce rankings different from the 'optimal' with a very high probability.
On the other hand, you know that some languages are bad or better at some issues. Likewise, you intuitively know that some MCDM methods are better at some issues.
We see that some languages are generally strong, widely used and adopted (eg English, etc.). We see that some MCDM methods are widely adopted and can be used in most conditions (For example TOPSIS, PROMETHEE etc.)
Mr. Munier,
As far as I understand from the ideas you shared in R.G., you have an alternative solution suggestion. In your opinion, the most suitable MCDM method can be chosen over an indirect proxy criterion. Accordingly, MCDM methods can be compared in relation to a proxy ranks. At this point, I would like to emphasize that I have similar thoughts to yours.
But the critical issue here is how to achieve this. So how will this proxy rank be achieved? Will it be artificially created by DM? Or should it be a completely real-life ranking with no DM interference? The second is the better idea in my humble opinion. We may also be thinking differently on this issue.
In other words, proxy ranking should be taken directly from real life and should be away from DM's subjective maneuvering area. In my opinion, this ranking, which is external, proxy and independent, can be solved more easily if it is taken directly from real life. Thus, it becomes clear which one of the MCDM results captures real life better. It should be emphasized again that the critical issue here is the question of how to find a real-life proxy ranking that D.M. is absolutely not involved in and cannot change.
There is a high effort behind over 100 MCDM method proposals. Because not choosing the best alternative creates an alternative cost. Therefore, I would like to state that I find this question and solution approach of Mr. Munier very meaningful and valuable.
Dear Nicholas
I don't know about the value-focused method, perhaps you could briefly explain what you mean.
I agree 100 % with what you said about the AHP.
Some weeks ago here in RG somebody asked about which are the alternatives for AHP. I believe there is an alternative: Don't use it!
Dear Mahmut
As it is known, the best MCDM method is either not available or it is a very difficult puzzle to know.
NM- We certainly can’t speak that in all cases method A is better than method B, but for a specific case I believe that this can be asserted founded on how well the A or B methods model the problem the best.
Of course, the compliance of only one requirement is not enough. The trick is to comply with as many requirements as possible.
This problem concerns many interdisciplinary applied sciences. So please let me make a metaphor for common understanding. Let's start with a similar question: What is the best language? (English, French ... et al.) The common impression is that, at first glance, the answer is relative.
NM - This is a good example, however if a language is good enough for the people who created it and serves well for whatever is needed, for this people this is the best language, as is yours or mine. The fact that we use English for connecting withy people around the world is another issue, it is a matter of convenience.
The purpose of languages is the same. The purpose of MCDM methods is the same.
NM -Not really. MCDM methods have a very clear purpose: to find the best solution of a problem, while the purpose of language is to communicate
Languages try to model reality with symbols. MCDM does this with numerical symbols. Languages express the same thing with high similarity. MCDM methods also often produce similar rankings.
NM -MCDM methods produce similar, but not identical rankings because subjectivity, but them should be identical. Languages may express the same thing in different ways, even there are some ideas that have a meaning in one language but have no translation in another, while MCDM uses always the same way; mathematics.
Languages evolve over time to best express a reality. MCDM methods are constantly being developed to find the best alternative. It is difficult to compare languages on a comprehensive and common objective measure of value that they create. It is difficult to compare MCDM methods on the basis of an objective value they create.
NM- In my opinion, the comparison is not based on what the MCDM methods create. It is based of how well they model and solve a problem
Languages can express a 'meaning' well or not. MCDM rankings likewise produce rankings different from the 'optimal' with a very high probability.
NM- How do you know that the probability to produce an optimum result is high?
In MCDM is quite the opposite, since this probability is mostly inexistent. Most researchers agree that it is impossible to achieve optimality in multi objectives due to opposite criteria. It is like you want to optimize Good or Bad. It is one or the other, or a balance, which is what MCDM methods do
On the other hand, you know that some languages are bad or better at some issues. Likewise, you intuitively know that some MCDM methods are better at some issues.
NM- Yes, this is true
We see that some languages are generally strong, widely used and adopted (eg English, etc.). We see that some MCDM methods are widely adopted and can be used in most conditions (For example TOPSIS, PROMETHEE etc.)
NM- Also true, but languages are most used because many circumstances such as historical, commercial, literature, etc.
Mr. Munier,
As far as I understand from the ideas you shared in R.G., you have an alternative solution suggestion.
NM- Well, it is not a solution, it is a hypothesis grounded on logic. For whatever activity people uses the tool that matches their requirements. You don’t use a shovel to excavate for a swimming pool but a back hoe excavator. The shovel can manage small jobs, requiring time and little effort, while the machine can do it rapidly, without effort.
In your opinion, the most suitable MCDM method can be chosen over an indirect proxy criterion.
NM- I suggested that some time ago, to compare results, not to compare abilities or capacities
Accordingly, MCDM methods can be compared in relation to a proxy ranks. At this point, I would like to emphasize that I have similar thoughts to yours.
NM- I am glad that we share the same opinion, but remember that to build a proxy rank, you need a tool thar best represent reality, with which we come back to the same initial point
But the critical issue here is how to achieve this. So how will this proxy rank be achieved? Will it be artificially created by DM? Or should it be a completely real-life ranking with no DM interference? The second is the better idea in my humble opinion. We may also be thinking differently on this issue.
NM- I don’t have the answer for that. Simple reasoning suggests that using a tool that considers most characteristics of a scenario, using original and reliable data, without the DM participation, and that can model it, is better than another tool that does not.
In other words, proxy ranking should be taken directly from real life and should be away from DM's subjective maneuvering area. In my opinion, this ranking, which is external, proxy and independent, can be solved more easily if it is taken directly from real life.
NM- Yes, but if you knew which the real ranking is, you don’t need MCDM
Thus, it becomes clear which one of the MCDM results captures real life better.
NM- You could do that if you knew which the real result is, but you don’t, even ignore if there is a solution.
It should be emphasized again that the critical issue here is the question of how to find a real-life proxy ranking that D.M. is absolutely not involved in and cannot change.
NM- In my humble opinion, one must begin comparing methods that do not use any subjectivity by means of subjective weights, assumed thresholds or assuming something that can’t be proved. From that very small number of such MCDM methods, you can say that the best for a certain problem is that which best represents it by modelling.
You can’t do serious work when you modify real data with such subjective appreciations like using pair-wise comparisons. What you do in such cases is to produce a result, mathematically correct, but that does not represent the real problem, it solves a problem that exists only in your mind. What is it good for?
There is a high effort behind over 100 MCDM method proposals. Because not choosing the best alternative creates an alternative cost. Therefore, I would like to state that I find this question and solution approach of Mr. Munier very meaningful and valuable.
NM- Thank you very much for your comments
Dear Dr. Munier,
Thank you for an inspiring response. I respect that it takes time and consideration to write such a helpful reply. It means a lot to me to know you really hear what I was conveying and responded so accordingly. I am honored by your efforts. Thank you.
NM- We certainly can’t speak that in all cases method A is better than method B, but for a specific case I believe that this can be asserted founded on how well the A or B methods model the problem the best.
Of course, the compliance of only one requirement is not enough. The trick is to comply with as many requirements as possible.
MB- I agree that there is no absolute superiority between the methods. It is a fact that some methods are more successful, adaptive and suitable models depending on real life applications. Some methods are more successful when the number of alternatives is low and some methods are more successful when they are high. The performance of some is affected by the number of criteria, while others are less affected. Some methods perform 'mediocre' in most cases, regardless of the number of alternatives and criteria. Some methods underperform in some cases, for no apparent reason. Some methods are successful because they use min-max normalization, for example, while some methods are successful because they do not use normalization. Some methods produce similar results because they use similar mathematical formulas. On the other hand, some methods interestingly produce the same results, although they are completely different in formulation. It is very difficult to explain the reasons for this. We can say that the dynamic relationship (not static) between real life and MCDM models is sometimes very complicated and sometimes extremely simple.
Perhaps the best MCDM method is a dynamic method that changes itself according to circumstances. Again, to make a metaphor, it is like a super vehicle that can travel both in the air, on land and in the sea. (maybe even in space). An easier choice is to use the MCDM method that best suits the situation and conditions.
I agree with you that the MCDM results are not 'optimal'.
NM- I suggested that some time ago, to compare results, not to compare abilities or capacities
MB- Data tools are available to compare abilities or capacities. But as far as I know there is no direct tool yet to compare results.
NM- Yes, but if you knew which the real ranking is, you don’t need MCDM
NM- You could do that if you knew which the real result is, but you don’t, even ignore if there is a solution.
MB- The human and mouse metaphor that you have emphasized elsewhere before was a good explanation. As is known, mice are used instead of humans in laboratory experiments. Because human and mouse genetics are similar to some extent. As such, comparative experiments can be performed on another ranking (genetically) similar to MCDM ranking. For example, MCDM methods can be compared on this criterion. I agree with this approach to a certain extent. But in some areas of application this 'mouse' is different. Or that thing may not be a mouse but a partial 'parallel universe'. Perhaps the best recipe could be a 'shadow universe' that grows and shrinks.
NM- In my humble opinion, one must begin comparing methods that do not use any subjectivity by means of subjective weights, assumed thresholds or assuming something that can’t be proved. From that very small number of such MCDM methods, you can say that the best for a certain problem is that which best represents it by modelling.
MB- Dear Dr. Munier, please correct me if I'm wrong. Subjectivity as a dilemma is both the defect and the reason why MCDM methods exist. When you remove this subjectivity, maybe MCDM will disappear gradually. We do not know if this extinction will later translate into a better existence, as in some living things. And we all actually sense this. On the other hand, I agree with you that some MCDM methods with less subjective participation of the decision maker are more suitable for comparability.
As for my opinion on subjective weights, I can say the following. Subjective methods actually reduce the existing subjectivity to a certain extent. So we cannot deny that these methods strive to bring subjective results closer to objectivity. It is also admirable that they care about collective decisions. Yes, that's their strength, because the right people make the right decisions. But the weak point of these methods is not the techniques they use. It is how they choose between the wrong decision makers and the right decision makers. Experts choose criteria or weight coefficient, but is the expert chosen correctly? Real arbitrariness or subjectivity, I think, in this questionable area.
Dear Mahmut.
MB- Thank you for an inspiring response. I respect that it takes time and consideration to write such a helpful reply. It means a lot to me to know you really hear what I was conveying and responded so accordingly. I am honored by your efforts. Thank you.
NM. When one receives comments like yours, I consider that those are worth of spending time thinking about them and answering. Thank you for your invaluable contribution.
MB- I agree that there is no absolute superiority between the methods. It is a fact that some methods are more successful, adaptive and suitable models depending on real life applications. Some methods are more successful when the number of alternatives is low and some methods are more successful when they are high.
NM – We are going always to the same concept. You talk that some methods are more successful than others, But how do we measure that successfulness? Based on what? I agree that some methods are more adaptive depending on real-life applications.
MB- The performance of some is affected by the number of criteria, while others are less affected. Some methods perform 'mediocre' in most cases, regardless of the number of alternatives and criteria. Some methods underperform in some cases, for no apparent reason.
NM- In reality I don’t see why performance may be affected by the number of alternatives or criteria. The algorithms do not depend on their number, and if they do, they are wrong. In my opinion that concept is important in probabilistic cases or in simulation, but not in MCDM. If an algorithm is sound it does not matter that repeats the same operation 4 or 400 times.
MB- Some methods are successful because they use min-max normalization, for example, while some methods are successful because they do not use normalization.
NM – I agree with you that the normalization method used may lead to different results, and I still ignore the reasons. In addition, normalization is a must in all methods, because if not, you are comparing different objects with different units. You can compare apples with gasoline when both values are without units, because then, you are comparing just decimal numbers.
MB- Some methods produce similar results because they use similar mathematical formulas. On the other hand, some methods interestingly produce the same results, although they are completely different in formulation.
NM – I don’t think that formulation is involved. If you use the same data and use different mathematical algorithms to process them, the result should be the same. That is, you can use the algorithm of the sum by adding 5 times the number 2 and you get the same result by multiplying 2 by 5
MB- It is very difficult to explain the reasons for this. We can say that the dynamic relationship (not static) between real life and MCDM models is sometimes very complicated and sometimes extremely simple.
NM- No doubt, and I would add that they are complicated in static conditions, which by the way, is one of the drawbacks of all MCDM methods, which ignore time. An example, is when you need to build several buildings and select which, normally MCDM assumes that all buildings start and finish at the same time, and that their yearly completion rate is equal, no matter the years to build each one, and obviously it is not. This is easily corroborated in the construction business when it is known that the progress does not follow a right line but a logistic curve.
MB- Perhaps the best MCDM method is a dynamic method that changes itself according to circumstances.
NM- Yes, most probably
MB- Again, to make a metaphor, it is like a super vehicle that can travel both in the air, on land and in the sea. (maybe even in space). An easier choice is to use the MCDM method that best suits the situation and conditions.
NM- Agreed, but real-life not always performs as we wish.
MB- Data tools are available to compare abilities or capacities. But as far as I know there is no direct tool yet to compare results.
NM – This is relative. There many facts in life where results compared. For instance, if you need to forecast traffic in a highway, you know the result as of today by physical measurement. Then. you can use a mathematical model like the gravity model to predict that. Normally you correct exponents or coefficients in your model, that is you calibrate your model until it produces acceptable results. However, you have a guide, since you know the true result from direct measurement in the highway. That is, you know the result, but you can’t calibrate a MCDM method because you don’t know which the real result is.
MB- The human and mouse metaphor that you have emphasized elsewhere before was a good explanation. As is known, mice are used instead of humans in laboratory experiments. Because human and mouse genetics are similar to some extent. As such, comparative experiments can be performed on another ranking (genetically) similar to MCDM ranking.
NM- Yes, but you know the results on the mice.
MB- For example, MCDM methods can be compared on this criterion. I agree with this approach to a certain extent. But in some areas of application this 'mouse' is different. Or that thing may not be a mouse but a partial 'parallel universe'. Perhaps the best recipe could be a 'shadow universe' that grows and shrinks
NM – And how do you know the result in this parallel universe?
MB- Dear Dr. Munier, please correct me if I'm wrong. Subjectivity as a dilemma is both the defect and the reason why MCDM methods exist. When you remove this subjectivity, maybe MCDM will disappear gradually. We do not know if this extinction will later translate into a better existence, as in some living things. And we all actually sense this. On the other hand, I agree with you that some MCDM methods with less subjective participation of the decision maker are more suitable for comparability.
NM- I disagree, there are MCDM methods such as CRITIC and SIMUS that do not have subjectivity, and even with this advantage, we can’t say they are successful.
MB- As for my opinion on subjective weights, I can say the following. Subjective methods actually reduce the existing subjectivity to a certain extent.
NM- I fail to see how they reduce subjectivity, albeit in a certain extent I agree with you. It is evident that in Promethee there is subjectivity in establishing thresholds and functions, but these are the consequence of reasoning. Compare with AHP where rationality is inexistent and only exists intuition.
MB- So we cannot deny that these methods strive to bring subjective results closer to objectivity.
NM. I am not sure about of this. Objectivity is using real values
MB- It is also admirable that they care about collective decisions. Yes, that's their strength, because the right people make the right decisions.
NM- Who are right people? People with experience and know how? I could agree on this.
MB- But the weak point of these methods is not the techniques they use. It is how they choose between the wrong decision makers and the right decision makers. Experts choose criteria or weight coefficient, but is the expert chosen correctly? Real arbitrariness or subjectivity, I think, in this questionable area.
NM – That experts use criteria rationally is undeniable, and they can do that very efficiently if they study the problem and discover and model all its demands. And yes, I believe that experts can do that objectively, and really, it is not very difficult. They are not treating nebulous characterises of a scenario, they are looking as costs, demands, environmental limits, social issues, people suffering or enjoying the consequences of a projects. They don’t need to put any value. It is a matter of thinking, studying the project and forecasting potential inconvenients.
It is like preparing your luggage for a trip, you mentally can recreate all possible aspects that can affect your trip, and develop the corresponding criteria, that is, tickets in different means, food, hotels, entertainment, prescriptions, travel insurance, local currency, time away from your job, drivers’ licence, etc. As a matter of fact, many years ago I didn’t do properly this list, I travelled to Japan, and I did not find if my Canadian driver licence was valid in that country.
It wasn’t, and I had to spend time and money in Tokyo and performing some tests, to get a drivers permit …...
Dear Dr.Munier,
NM – We are going always to the same concept. You talk that some methods are more successful than others, But how do we measure that successfulness? Based on what? I agree that some methods are more adaptive depending on real-life applications.
MB- You can see this when you compare the methods according to the strength of their similarity with real-life rankings. For example, the PROMETHEE method is one of my favorites for my application area. Accordingly, I agree with your general approach that there is enough effort on talent and capacity, but now it is time to make comparisons on results.
NM- In reality I don’t see why performance may be affected by the number of alternatives or criteria. The algorithms do not depend on their number, and if they do, they are wrong. In my opinion that concept is important in probabilistic cases or in simulation, but not in MCDM. If an algorithm is sound it does not matter that repeats the same operation 4 or 400 times.
MB- Many MCDM methods are affected to some extent by the number of alternatives or criteria. MCDM is the common living space of objectivity and subjectivity. In addition to the algorithm, there are also cases where DM involvement has a high impact, such as the assumption used, threshold values, preference function, preferred normalization type, and weighting type.
For me, the result is important. The following can be said for some MCDM methods with less DM effect. Some MCDM methods model real life better when the number of alternatives increases. For some, this is the opposite. For example, let's assume that we have obtained two different rankings with TOPSIS, with 25 alternatives and 100 alternatives, provided that the same criteria are used. Ideal positive and ideal negative values may be completely different for 25 and 100 alternate rankings. This of course changes the strength of the ranking results. In other words, TOPSIS is affected by the number of alternatives and the success of modeling reality is likely to change. Better ideal positive values and worse ideal negative values can be obtained from higher number of alternate rankings. This means getting a higher quality ranking if you have a goal to capture reality. It is more reliable when the sample is high in statistics, and it can be said that it varies according to this method in MCDM.
NM - I agree with you that the normalization method used may lead to different results, and I still ignore the reasons. In addition, normalization is a must in all methods, because if not, you are comparing different objects with different units. You can compare apples with gasoline when both values are without units, because then, you are comparing just decimal numbers.
MB- For members of the classical utility school, such as TOPSIS, using normalization is an important requirement. Of course, I agree with your opinion for this school. But correct me if I'm wrong: European outranking schools often calculate based on their superiority in binary comparison. Therefore, they do not feel the need for normalization. For example, the PROMETHEE procedure does not require or impose normalization, but uses a preference function. I know that many outranking schools such as QUALIFLEX, ORESTE and even ELECTRE do not force the data to normalize. Do you think preference functions are a kind of 'partial normalization' in accordance with the nature of ranking problems?
NM – I don’t think that formulation is involved. If you use the same data and use different mathematical algorithms to process them, the result should be the same. That is, you can use the algorithm of the sum by adding 5 times the number 2 and you get the same result by multiplying 2 by 5
MB- You have very well described the rule that should be. In fact, there may be high correlations between the results. But differences in approach (utilitarian or outranking school) or a little different preference produce different results. For example, WSM and SAW methods can produce different results even though they have the same calculation procedure. Because they use different types of normalization. This reveals that the data is not transformed equally in certain scales. This can be a critical issue when looking for the best alternative.
NM- No doubt, and I would add that they are complicated in static conditions, which by the way, is one of the drawbacks of all MCDM methods, which ignore time.
MB- Your approach is absolutely correct in this matter. Time and risk factors should be taken into consideration in MCDM methods.
NM- Agreed, but real-life not always performs as we wish.
MB- Yes, I often see this interesting situation in experimental studies. MCDM methods sometimes show very high similarity with real life, on the other hand, sometimes it is lower.
NM- That is, you know the result, but you can’t calibrate a MCDM method because you don’t know which the real result is.
MB- We do not know the real result and I agree with you. But it may be possible to calibrate the MCDM method using a similar external ranking. Undoubtedly, this is a subject for which you are more competent.
NM- Yes, but you know the results on the mice.
Yes we know.
NM- And how do you know the result in this parallel universe?
From real life. Here are two facts / events for an alternative. While the first event is obtained (modeled) indirectly with MCDM, the second event is obtained directly from real life.
NM- I disagree, there are MCDM methods such as CRITIC and SIMUS that do not have subjectivity, and even with this advantage, we can’t say they are successful.
MB- Actually, we are of the same opinion. Let me explain please. Which method is successful is a completely different matter. The issue here is that it is easier to compare MCDM methods with more reliable objectivity. That is, it is more reliable to compare methods with relatively low DM involvement. For example, TOPSIS and SAW method are more comparable methods. Comparison results are more valid and reliable.However, can you say similar to compare AHP-BWM with the same data? On the same subject, for example, expert opinion in the USA and expert opinions in Germany may be different.
Dear Mahmut (please drop the Dr, use my name, as I use yours, thank you}
NM – We are going always to the same concept. You talk that some methods are more successful than others, But how do we measure that successfulness? Based on what? I agree that some methods are more adaptive depending on real-life applications.
MB- You can see this when you compare the methods according to the strength of their similarity with real-life rankings.
NM- Sorry, this is new for me, I have never heard of real-life rankings, at least in MCDM. Now if you mean that real-life ranking is a measure of what you achieve from the personal point of view, such as happiness, I can understand. Suppose that for a certain scenario you use TOPSIS and PROMETHEE, and you get a ranking A>C>B with the first, and C>A>B with the second. How can you compare them? Based on what real-like rankings. I am not sating that you are wrong, simply, I don’t understand.
MB - For example, the PROMETHEE method is one of my favorites for my application area. Accordingly, I agree with your general approach that there is enough effort on talent and capacity, but now it is time to make comparisons on results.
NM – This attempt has been done for years, unfortunately without success
NM- In reality I don’t see why performance may be affected by the number of alternatives or criteria. The algorithms do not depend on their number, and if they do, they are wrong. In my opinion that concept is important in probabilistic cases or in simulation, but not in MCDM. If an algorithm is sound it does not matter that repeats the same operation 4 or 400 times.
MB- Many MCDM methods are affected to some extent by the number of alternatives or criteria. MCDM is the common living space of objectivity and subjectivity. In addition to the algorithm, there are also cases where DM involvement has a high impact, such as the assumption used, threshold values, preference function, preferred normalization type, and weighting type
NM- No MCDM method should be affected by the number of criteria. It does not matter if you have a 5 x 6 matrix or a 500 x 600. The later can take more time to solve, but the process is the same.
However, I agree with you if we are talking about simulation or a survey, were size is important, but those are not MCDM processes.
Of course, the DM involvement has a high impact, and that is precisely the reason by which the DM must select the criteria, establish their attributes and objective values and constrains, but other than that he/she has no business in modifying real data and establishing assumptions. It is at the end of the process, when the DM has the result based on real, solid, objective grounds, that he must use his experience, know how and common sense to accept the result, modify it, even deleting or adding new criteria or values, or simply reject it.
An explanation of why he can’t use his subjective values is that he does not know how they will impact and combine with others, and in addition, because these corrected values are relative, since another DM may have different ones for the same problem and method.
However, I agree with you that in case of PROMETHEE, the DM uses a rational approach based on his/her experience, something that I believe happens in TOPSIS, since the selection of the best distance is arbitrary, that is why Manhattan or Euclidean.
I also agree in normalization because if he selects for instance Max/Min he does do that looking for discrimination is the scores. The same for weighting type as long as they are objective. Subjective weighting is completely arbitrary and there is nothing that can justify its use.
MB- For me, the result is important. The following can be said for some MCDM methods with less DM effect. Some MCDM methods model real life better when the number of alternatives increases. For some, this is the opposite. For example, let's assume that we have obtained two different rankings with TOPSIS, with 25 alternatives and 100 alternatives, provided that the same criteria are used. Ideal positive and ideal negative values may be completely different for 25 and 100 alternate rankings. This of course changes the strength of the ranking results. In other words, TOPSIS is affected by the number of alternatives and the success of modeling reality is likely to change. Better ideal positive values and worse ideal negative values can be obtained from higher number of alternate rankings. This means getting a higher quality ranking if you have a goal to capture reality. It is more reliable when the sample is high in statistics, and it can be said that it varies according to this method in MCDM.
NM- If you solve two scenarios one with 25 alternatives and the other with 100 and subject to the same criteria, of course you will get different results, because they are different problems.
MB- For members of the classical utility school, such as TOPSIS, using normalization is an important requirement.
NM – That requirement is mandatory for all MCDM methods.
In TOPSIS, normalization is also required. When you compute the R+ for each row you compute differences, that are added to the calculation done for another row. If the first row refers say to ‘Cost’ and the second refers to say ‘Slope of the road’, how can you add them if they are not normalized?
MB- Of course, I agree with your opinion for this school. But correct me if I'm wrong: European outranking schools often calculate based on their superiority in binary comparison. Therefore, they do not feel the need for normalization. For example, the PROMETHEE procedure does not require or impose normalization, but uses a preference function. I know that many outranking schools such as QUALIFLEX, MAPPAC, ORESTE and even ELECTRE do not force the data to normalize. Do you think preference functions are a kind of 'partial normalization' in accordance with the nature of ranking problems?
NM- In Promethee you compare alternatives A and B according to one criterion and get a value that is further multiplied by the criterion weight. You do that for each criterion, but then, you must to add-up those values, and for that you need to have them normalized, because criteria may have different units.
MB- You have very well described the rule that should be. In fact, there may be high correlations between the results. But differences in approach (utilitarian or outranking school) or a little different preference produce different results. For example, WSM and SAW methods can produce different results even though they have the same calculation procedure. Because they use different types of normalization. This reveals that the data is not transformed equally in certain scales. This can be a critical issue when looking for the best alternative.
NM. Maybe, I am not sure about this. For instance, the SIMUS method gives the same results if you use total sum, the largest value or Euclidean, but gives different values for Max/Min. In addition, I remind you that you normalize data before you enter in a process.
MB- Yes, I often see this interesting situation in experimental studies. MCDM methods sometimes show very high similarity with real life, on the other hand, sometimes it is lower.
NM – Again. How do you prove that there is similarity with real-life results if you don’t know it?
MB- We do not know the real result and I agree with you. But it may be possible to calibrate the MCDM method using a similar external ranking. Undoubtedly, this is a subject for which you are more competent.
NM -Probably you refer that you can use a proxy result that might have similitude with the real one (if it exists). When I proposed that years ago, I was using SIMUS that does not have any subjectivity, as in real-life. Subjectivity is a human feature and in Nature for instance, which produces the best result of anything, subjectivity is absent. But it does not mean that SIMUS is the only one or the best for this job, its main advantage is lack of subjectivity, and maybe there are other MCDM methods with the same property.
NM- And how do you know the result in this parallel universe? By the way, when you used this expression, I understood that you were referring as the cosmological hypothesis of parallel worlds
MB - From real life. Here are two facts / events for an alternative. While the first event is obtained (modeled) indirectly with MCDM, the second event is obtained directly from real life
NM – Clever, but how do you know the real-life, say Nature, consider also other alternatives as in MCDM?
Or in our world, if you know about a similar project that physically used processes or materials as yours, how do you know that it is the right solution? For instance, consider two similar buildings in a certain area, with the same orientation and height; one of them already built, and the other in construction. People already leaving in the first complain about poor ventilation and not enough walls insulation. Are you to take that real life construction as a model for the second one?
NM- I disagree, there are MCDM methods such as CRITIC and SIMUS that do not have subjectivity, and even with this advantage, we can’t say they are successful.
MB- Actually, we are of the same opinion. Let me explain please. Which method is successful is a completely different matter. The issue here is that it is easier to compare MCDM methods with more reliable objectivity. That is, it is more reliable to compare methods with relatively low DM involvement. For example, TOPSIS and SAW method are more comparable methods. Comparison results are more valid and reliable.
NM- On which they are comparable? They are totally different. Maybe the only resemblance is that both use weights. SAW is the simplest MCDM method and it is only a weighted sum, while TOPSIS is based on distances.
MB- However, can you say similar to compare AHP-BWM with the same data? On the same subject, for example, expert opinion in the USA and expert opinions in Germany may be different.
NM- In my opinion, you could but you shouldn’t, because AHP fabricates data while BWM starts from given data and arbitrarily selects which is the best and the worst criteria.
Experts’ opinion is not a matter of nationalities. An expert on soils for instance, has the same knowledge around the world, however, each one may have more knowledge and expertise in the geological nature of a certain area.
Dear Mahmut, this is really a very interesting subject
Dear Nolberto,
NM- Sorry, this is new for me, I have never heard of real-life rankings, at least in MCDM. Now if you mean that real-life ranking is a measure of what you achieve from the personal point of view, such as happiness, I can understand. Suppose that for a certain scenario you use TOPSIS and PROMETHEE, and you get a ranking A>C>B with the first, and C>A>B with the second. How can you compare them? Based on what real-like rankings. I am not sating that you are wrong, simply, I don’t understand.
MB- This is not very new to your. You used the mouse-human analogy to better explain this topic. On the other hand, you can suggest SIMUS for TOPSIS and PROMETHEE comparison.
NM- No MCDM method should be affected by the number of criteria.
NM- If you solve two scenarios one with 25 alternatives and the other with 100 and subject to the same criteria, of course you will get different results, because they are different problems.
MB- No Comment.
MB- For members of the classical utility school, such as TOPSIS, using normalization is an important requirement.
NM – That requirement is mandatory for all MCDM methods.
MB- As you know, the 'usual preference function' is widely used in PROMETHEE procedure. You may not need normalization when you use this preference function (remember: It always assigns 1 if the value d> 0 and 0 if the value d ≤ 0.). Because -for this preference function- the principle of superiority in binary comparisons is valid in PROMETHEE. Even if you create the first decision matrix with different units, you will eventually create it with values 0 and 1. So I think normalization is not necessary. A real life analogy can be made on this subject. In football competitions, the winning team gets 3 points and the loser 0 points. There are also studies that calculate the PROMETHEE method without normalization. Palczewski and Salabun (2019) I can give as an example. One more example: Maybe you are hearing it for the first time, for example, Faire Un Choix Adéqua-FUCA method can calculate without data normalization.
Yes, I agree with you that there are many methods that have to be used normalization. However, this requirement may not be mandatory for all MCDM methods.
NM- In Promethee you compare alternatives A and B according to one criterion and get a value that is further multiplied by the criterion weight. You do that for each criterion, but then, you must to add-up those values, and for that you need to have them normalized, because criteria may have different units.
MB- Correct me if I'm wrong. You may not need normalization when you use the 'usual preference function'.
NM -Probably you refer that you can use a proxy result that might have similitude with the real one (if it exists).
MB- Yes that's right. What's more, there is a confusing part: This proxy result is directly a real-life product.
NM – Clever, but how do you know the real-life, say Nature, consider also other alternatives as in MCDM?
MB- Finding a proxy result that can be similar to the real result can take a lot of creativity and effort. This is a problem that is not easy for some research areas. Finding a generalizable solution is probably more difficult.
Dear Mahmut
NM- Sorry, this is new for me, I have never heard of real-life rankings, at least in MCDM. Now if you mean that real-life ranking is a measure of what you achieve from the personal point of view, such as happiness, I can understand. Suppose that for a certain scenario you use TOPSIS and PROMETHEE, and you get a ranking A>C>B with the first, and C>A>B with the second. How can you compare them? Based on what real-like rankings. I am not sating that you are wrong, simply, I don’t understand.
MB- This is not very new to your. You used the mouse-human analogy to better explain this topic. On the other hand, you can suggest SIMUS for TOPSIS and PROMETHEE comparison.
NM- I don’t remember using the mouse-human analogy, but if you say so, I believe you.
If I did, probably I was referring that you can extract conclusions from the mouse to be applied to a human being. If it is so there is a difference with MCDM because in the analogy we are comparing two beings that are alive and not generated by any algorithm. Even thou, I fail to understand how this is related to a ranking.
Yes, I suggested SIMUS for comparison as you say, because if SIMUS don’t use any subjectivity, it could the most closed to reality, that uses no subjectivity. Therefore, the closer to SIMUS of TOPSIS or PROMETHEE results the better. But this is only a hypothesis of mine.
An analogy comes to my mind. Consider a river between two zones A and B. It is formed by right sections followed by twisted sections. Which do you think is the reason for that? I am not a geophysicist but I can guess that perhaps thousands of years ago, the Great Architect, Mother Nature, ‘had to decide’ the best route to drain a lake located in A, and based on hundreds of intervening features, due to soil features, slopes, weather, rains, forests, soil temperature, air dryness, etc., that were real factors.
Mother Nature managed to get the course of the river that better harmonized all features, and considering all those factors Nature did not use decision-makers, experts or assumed assumptions. That is, she took the existent elements and combined them to reach a compromise solution. Later on, man came, and changed different aspects, building dams, diverting water for irrigation, etc.
In my hypothesis, there is a resemblance between MCDM, in the sense that if we want to develop and algorithm that mimic the shape of a river, we should use only objective data, as Nature did. After the result is known, and similarly with the anthropogenic actinon in the river, the DM can also make changes. Consequently, I gather, that the closest the MCDM method to total objectivity, the more chances that the DM can approximate reality, and does it what SIMUS does. I am not saying that for that it is the best method, most possibly it is not, but since it follows the same objective principles of Nature, these are hints that it is correct.
NM- If you solve two scenarios one with 25 alternatives and the other with 100 and subject to the same criteria, of course you will get different results, because they are different problems.
MB- No Comment
NM – Does it mean that you consider that they are not different problems?
MB- For members of the classical utility school, such as TOPSIS, using normalization is an important requirement.
NM – That requirement is mandatory for all MCDM methods.
MB- As you know, the 'usual preference function' is widely used in PROMETHEE procedure. You may not need normalization when you use this preference function (remember: It always assigns 1 if the value d> 0 and 0 if the value d ≤ 0.). Because -for this preference function- the principle of superiority in binary comparisons is valid in PROMETHEE. Even if you create the first decision matrix with different units, you will eventually create it with values 0 and 1.
NM – I suggest reading ‘Influence of various normalization methods in PROMETHEE II: an empirical study on the selection of the airport location Krzysztof Palczewskia , Wojciech Sałabuna* aDepartment of Artificial Intelligence Methods and Applied Mathematics’ and’ Investigating the Effect of Normalization Norms in Flexible Manufacturing Sytem Selection Using Multi-Criteria Decision-Making Methods’ Prasenjit Chatterjee*,1 and Shankar Chakraborty2
So I think normalization is not necessary. A real life analogy can be made on this subject. In football competitions, the winning team gets 3 points and the loser 0 points.
NM – Of course, they are using the same units.
MB- There are also studies that calculate the PROMETHEE method without normalization. Krzysztof Palczewski et al. / (2019) I can give as an example. One more example: Maybe you are hearing it for the first time, for example, Faire Un Choix Adéqua-FUCA method can calculate without data normalization.
NM- You are right, I don’t know this method, but perusing in the document above mentioned, its says in page 561 ‘Second, the selected method should be able to deal with objectives having significantly different magnitudes, which essentially requires a normalization step.’
-Probably you refer that you can use a proxy result that might have similitude with the real one (if it exists).
MB- Yes that's right. What's more, there is a confusing part: This proxy result is directly a real-life product.
NM – Clever, but how do you know the real-life, say Nature, consider also other alternatives as in MCDM?
MB- Finding a proxy result that can be similar to the real result can take a lot of creativity and effort. This is a problem that is not easy for some research areas. Finding a generalizable solution is probably more difficult. Could be co nsidered a proxy of reality.
NM- Yes, I agree
MB- For members of the classical utility school, such as TOPSIS, using normalization is an important requirement.
NM- As far as I know there is no utility in TOPSIS, however, there are some papers that have used TOPSIS with utility, apparently to avoid rank reversal.
MB .So I think normalization is not necessary. A real life analogy can be made on this subject. In football competitions, the winning team gets 3 points and the loser 0 points. There are also studies that calculate the PROMETHEE method without normalization. Krzysztof Palczewski et al. / (2019) I can give as an example. One more example: Maybe you are hearing it for the first time, for example, Faire Un Choix Adéqua-FUCA method can calculate without data normalization.
Yes, I agree with you that there are many methods that have to be used normalization. However, this requirement may not be mandatory for all MCDM methods.
NM- In PROMETHEE you compare alternatives A and B according to one criterion and get a value that is further multiplied by the criterion weight. You do that for each criterion, but then, you must to add-up those values, and for that you need to have them normalized, because criteria may have different units.
Dear Nolberto,
Even if the criteria have different units, you may not need normalization when you use the 'general preference function'. Because in the next step, the decision matrix will be created with only 0 or 1 values. (remember: It always assigns 1 if the value d> 0 and 0 if the value d ≤ 0.) In my opinion, there is no need to lose information twice. I recommend you the first article you suggested. Because the article has both normalized and non-normalized PROMETHEE results.
As it is known, deviations based on binary comparisons are determined in step 1 in PROMETHEE. In other words, the 'difference' between a and b evaluations in each criterion is specified in this step. I think the strongest side of this method is the 'difference' approach in this first step. In the past, some outranking methods may have been criticized for losing knowledge. But if one pays attention to the MCDM paradigm there is an intense smell of 'competition'. When your goal is to find the 'best alternative', it would be a more assured approach to clarify the dual advantages between alternatives. So it is more important not how superior you are, but whether you are superior or not. I do not know which is the best MCDM method. But in my opinion, the method that simply guarantees clarification of the advantages in binary comparison is a very suitable MCDM method for many problems.
In addition, it is a fact that normalization methods seriously affect the results of MCDM rankings. For example, with min-max normalization, an alternative that ranks first can be placed last with another normalization. This situation brings to mind the question of "information loss". The 'general preference function' in PROMETHEE or the 'ranking rank value' in methods such as ORESTE seems to be a serious alternative to normalization.
I am grateful to you once again, dear Nolberto, who increased my knowledge and curiosity and added different dimensions to my perspective.
Suggest reviewing this paper for MCDM methods:
https://www.sciencedirect.com/science/article/abs/pii/S092583881732827X