Dear Nolberto:

could you please provide me more information about your statement " the criterion with the highest weight is the most important, which by the way it is easily demonstrable that said assumption is a fallacy ".

In my opinion, another problem with MCDM methods is the lack of a proper benchmarks to evaluate them. As long as I know, there are no problems for which the "true" ranking is available. In such situation, unless an alternative dominates another one (in the Pareto sense), any ranking can be valid.

Regards

David

Dear David

Thank you for answering my question and for formulating this very interesting question

First of all, when subjective weights are used, as determined in AHP and employed in other methods, it is necessary to take into account that they are not really weights, but trade-off values, as Saaty also says in one of his papers published in 2008, definition that is also shared by most researchers.

Consequently, they inform how the relationships between criteria changes when one of them change. It is the classical ‘take from Bob to give to Paul’, or, if we are comparing two criteria such as quality and prices in restaurants, you may sacrifice quality to gain a better price or vice versa, but it does not mean than price is more important than quality.

As a matter of fact, in comparing with another restaurant it may be that it is better to sacrifice price to get a better quality. This demonstrate that you can’t pairwise compare criteria if you do not consider the alternatives. Another AHP fallacy.

Consequently, trade-ff values do not represent criteria weights.

Secondly, even if they were weights, they would be only expressing the relative importance between criteria

However, even in this case, they are not fit to evaluate alternatives, because for that purpose it is necessary to use the quantity of information that each criterion provides, based on its performance values. The higher the discrimination the better. This is not only common sense but it is supported by Shannon (1948) theorem on the information that every message or criterion provides, which by the way is the foundation of Information Theory.

Remember that the so called ‘weights’ are obtained by the DM personal and then subjective preferences, then, they can be biased. Another DM may have different preferences.

However, you can determine REAL and OBJECTIVE WEIGHTS, if you elicit them using entropy. Because they have been obtained from the original data, they DO represent the quantity of information that each criterion carries. Consequently, the lower the entropy the better, because the maximum the information content, and you can then evaluate alternatives safely. You can use the entropy values to rank and compare criteria.

Suppose that the DM needs to evaluate three alternatives x1, x2 and x3 for a highway between city A and city B, and subject to 10 criteria, and that he has obtained, again, by personal preferences, that the highest weights correspond to ‘Cost’ and ‘Distance’ with 0.21 and 0.14 respectively, therefore, he decides that the most important criterion is ‘Cost’, while other criteria such as ‘Swamps crossed’ and ‘Crossing scenic areas’ are valued 0.02 and 0.01 respectively.

The ranking resulted as x1> x3> x2

Now he develops true weights using entropy, and doing the same exercise he gets that ‘Swamps crossed’ is now 0.327, and ‘Crossing scenic areas’ is 0.154, However, now ‘Cost is 0.001 and ‘Distance; 0.001. That is ‘Cost’ and ‘Distance’ are completely irrelevant. In fact, if he increases their weights nothing will happen.

These means that the DM should use ‘Swamps crossed’ and ‘Crossing scenic areas’ for alternatives evaluation, instead of ‘Cost’ and ‘Distance’.

As a matter of fact, actual systems perform sensitivity analysis using only one criterion. This is another fallacy, because they should consider all criteria simultaneously since all of them are related, as corresponds in a system

The ranking resulted as x3>x2>x1

These are not invented numbers. They respond to an actual work solved by TOPSIS and you can have a copy if you want.

You can see the radical difference between using true weights instead of using proxy weights

Do you understand why I said the actual procedure for doing sensitivity analysis is a fallacy? There are no mathematical grounds to support the idea that the highest the wright the better; it is only intuitive. I am not the only researcher that affirm this as Greco (2006) says ‘But, this kind of analyses has rather a theoretical interest than a practical one’.

I hope that my explanation satisfies you, if not, we can continue this conversation.

As per your second paragraph I wholly agree with you and I am afraid that there will be never one, because as you say we don’t know which the ‘true’ solution is.

What you say about that one alternative dominating other as Pareto, is true, the problem is that it can happen for one objective but very rarely for all of them. In that situation which do you choose? One way could be to work with the most important objective, but how do we know which is the most important?

When you solve a problem in LP you analyze the initial matrix and it gives you the scores for each alternative (Primal problem).

When you use SIMUS you get at the same time which are the objectives that conform the best solution.

At the same time LP solves a second problem (called Dual problem), based on the transpose of the initial matrix. Its results give de shadow prices, because they indicate the utility of each criterion regarding a certain objective

Combining the results of the Primal and the Dual the DM knows exactly which criteria he must use for a sensitivity analysis. In this way he can get a curve with quantitative values of the benefits that produce increasing a certain resource or criterion

Dear Nolberto:

thank you for your detailed explanation! We have some research (submitted) in the field of multiobjcetive optimization and how we can select "solutions of interest" from the Pareto front. We think this could be translated to MCDM problems.

As soon as we have a decision on the paper I'll contact you again (directamente por correo. Veo que estás en Valencia y eres de Argentina. Yo tambien :)

All the best

David

Gracias David

Mi correo es [email protected]

Yo acabo de volver de Argentina, donde di tres conferencias sobre este tema,.

No estoy en Valencia, sino que ahora vivo en Kingston, Canada, y estuve haciendo gestiones con varias universidades de Cordoba y Baires, para formar un grupo internacional para establecer nuevos paradigmas en la toma de dercisiones. Si te interesa eres muy bienvenido

Donde estas radicado?

It is very important to evaluate the reliability of MCDM method because once we take expert advice for the calculation of weight then there may be large variations which can affect the ranking of alternatives. Thus, it is important to evaluate the change of ranking of alternatives by changing the weight of criteria.

I would suggest that there is nothing ideal in any MCDM method thats why it is necessary to do sensitivity analysis.

Dear Saif

I agree that it is very important.

I also agree what you say about how the ranking changes when you change the weight of criteria.

Unfortunately, what is done today to this respect is meaningless, because changing the 'weight' (which is not such, but trade-offs), gives useless information, mainly because the so-called weight is not fit to evaluate an alternative, because it is not related with the performance values of each criterion.

(Shannon theorem).

The other factor is that only one criterion weight is changed, and this criterion has been selected arbitrarily, since it is assumed that the most important criterion is that with the highest weight. This is only intuition, and no mathematical theory supports it. In reality, all criteria that are relevant to the best alternative should be considered simultaneously, and no method does that.

In my opinion sensitivity analysis is not related with an ideal; it is necessary because the uncertainty of values used in the decision matrix

Dear Nolberto,

If I’m clear are you suggesting to change the weights of the criteria for a better sensitivity analysis instead of the weight of the alternative?

Dear Egbe

Not really.

What I am saying is that the way sensitivity analysis as performed nowadays, is not efficient and mainly based on false concepts and assumptions. I said nothing about alternative scores.

Sensitivity analysis, in my opinion, must rest on the following premises:

1. It must consider the variation of not one criterion arbitrarily chosen, but on the simultaneous variation of those that influence the solution. The reason is that normally not only one criterion conforms the selected alternative but many. Nowadays it is considered the variation of only one criterion and assuming that the others remain constant. This is a flagrant violation of the principles of a system, when all factors are related, and then changing one of them usually implies changing others.

2. The identification of the criteria or criterion (since in some problems the alternative depends only on one criterion, although this is rare), can’t be intuitive, as at present, based on the value of the respective ‘weight’. The identification must be based on scientific analysis

3. The influence of the criteria on the best selection, which is the base of the sensitivity analysis, is related with the importance of each criterion. This importance has nothing to do with the ‘weight’ of the criterion, but with its quantitative capacity to influence the best selection.

This capacity is given by the marginal value of each criterion. When you know this value for each of them, you can combine them all, and analyze the influence on the selected alternative. As a reminder, a rough definition of a marginal value is the effect that it produces in the output, in our case the objective, when the importance of the input, in our case, the criterion, is increased in one unit. If various criteria participate in the best selection, the objective will change as a function of increasing one unit to each criterion, and then adding up the marginal values, positives and negatives

4. Sensitivity analysis must be capable of considering exogenous factors than can act on the best alternative and quantitatively compute the different risks that these multiple variations pose for the best alternative.

None of this is done at present, consequently, there is a need to change the actual procedure

Okay Nolberto.

The way I conduct sensitivity analysis is to take out any parameter that contributes less to the model? Do you think this is the best way to go about sensitivity analysis?

Egbe

Yes, it could be, however, how do you know which are the criteria that contribute less?

I imagine that you are not considering the values of criteria with lesser their 'weights', because again, that means NOTHING

Nolberto,

I consider first, removing the criteria with less weights, increasing the criteria with more weights, or combining some alternatives to achieve a hybrid approach.

Dear Egbe

First of all, let me tell you that I enjoy this interchange with you. I wish many people could follow your lead. Please continue with it.

Subjective 'weights' as those obtained from AHP, DO NOT represent criteria relative importance, as I told you before, since they are trade-offs,therefore, you get nothing by removing or increasing them.

For the same token, you also get nothing by increasing criteria with higher weights. In addition, you don't know how many criteria participate in the best selection.

Now, if you work with objective real weights for criteria, such as those from statistics or entropy, then they represent criteria importance, and you can use them.

However, even in this case, you don't have a hint about which criteria consider.

It is necessary then to work with the real value for each criterion, and this is its MARGINAL values

This is at least my opinion and I would very much like indeed if more RG members challenge what I say, with valid arguments, of course.

Does it mean that we can't perform a rational and scientific sensitivity analysis?

Yes, we can, using a MCDM that allows for determination of marginal values. That method is Mathematical Programming, and it is in the Excel add-in of your computer, under the 'Solver' name

Dear Nolberto,

Thanks for the clarification. I have a better idea of what your thoughts are concerning sensitivity analysis. I always want to learn from experts in my field of engineering. Thanks for sharing your knowledge with me.

Dear Egbe

You are most welcome. Pls feel free to continue this discussion if there is some aspect that I could perhaps clarify