Don't run AHP. To begin, groups don't make decisions, they have emergent behaviors, and AHP doesn't apply. The math of this is game theory. Secondly, AHP isn't mathematically valid anyway. Third, weighted sum methods have very restrictive requirements, and they are almost always improperly used. If you want to use multiattribute ranking methods, learn the theory. The weights do not represent importances, they define indifference surfaces.

Here's the decision tree...

Dear Pietro

Thank you for posting your problem

I work with Linear Programming and SIMUS, which is based on LP. The difference is that the first gives, if it exists, the optimal solution but working with only one objective function. The second does not guarantee an optimal solution, but a derived from heuristics, that is, a satisfying one, but works with any number pf objectives.

Because your problem has only one objective, I solved it using LP.

Of course, I don’t know what your problems refer to, but since you provided the number of alternatives, criteria and sub-criteria, I used that data and simulated the problem with values at random. I also assumed at random values for the objective function.

LP does not use trees or multilevel decision structures, nor use any weights, because the matrix algebra of LP replaces criteria weights by a ratio in each iteration, which changes at each iteration.

Please see in the attached file how I modelled the problem. In there, the five alternatives are in columns and the clusters and criteria in rows. Since all alternatives are related with all criteria, I repeated them three times in order to consider the three clusters DEC, DES y DEC, that is, I repeated the criteria RDI, DP, APC, RUAC, LE, AFS, ACP AND ARP for each cluster, and I assume that each criterion has different performance values from one cluster to another.

I put random values for the correspondence between each alternative and each criterion in each cluster.

In addition, since I don’t know what those acronyms mean I have to invent the action, since each criterion is in reality an objective and has to be maximized or minimized.

Selection of alternatives

The attached file is in Excel, and I used the Excel add-in, called Solver, for this problem. You can see the solution in the solid black row with white values.

The alternative values are:

FTA=0.6977

ISA=0.7010

TPA= 0.9443

ISEP=0.4492

AM=0.0624

Since in this simulation I maximized the objective, the best choice appears to be TPA, followed by FTA, etc. This solution for this problem is optimal, since it can’t be improved, according to Pareto principle.

Of course, if you agree, you can use the dame procedure using your own and true values.

Criteria relative importance

At the same time, and automatically, Excel delivers the dual of this problem, that gives the marginal values of each criterion or shadow prices. These can be considered as the mathematical weights for each criterion, and indicate how much the objective function increases or decreases each time that we increase or decrease the original second term of each inequality. In the attached file you can find the relative mathematical and optimal ‘weight’ of each criterion.

The most important appears to be RUAC = 0.82 within cluster DEC, followed by ARP = 0.69 within cluster DES, followed by RDI= 0.60 within cluster DEN, and being the least significant APC = -0.13, within cluster DEN, and as that with a negative effect.

Observe that RUAC = 0.39 within cluster DEN, while there is another RUAC value, but for a different cluster (DEC). I don’t know if this makes sense or not, because I don’t know their meaning, but I am sure that you will be able to understand the difference.

It seems that the most important cluster is DEN, because it contains three criteria that have direct influence in the objective function, in comparison with the other two clusters.

Allowable range for criteria variation

It also gives the range that a criterion can vary, that is how much can we increase or decrease the second term to have no variation in the functional. This is the heart of the sensitivity analysis done by LP and SIMUS.

Group Decision-Makers

In the attached file there is a brief explanation of how to work with a Groyup.

Essentially, the SIMUS software is run and stops after each partial feasible result giving an objective value Z.

At that moment some DMs may want to make observations or experiments to see if the Z can be improved, and then changes are proposed and registered in the initial decision matrix.

The software is run again and the new Z is compared with the original. If the new one improves Z, the modifications are considered permanent, if not, rejected. It only takes a few seconds por the software to give new solutions

The software is run again ant the it stops at the second iteration. The same procedure is repeated until all iterations have been considered.

The great advantage of this method is that there is a quantitative value that allows for comparisons; therefore, preferences are omitted and acceptance is based on an optimal Z value.

I hope that this analysis can help you; if you have any doubt about this procedure don’t hesitate in contacting me.

By the way, SIMUS software is in the WEB and it is completely free for any person or institution.

Regards

Nolberto Munier

Dear M. Munier,

thank you very very much for your answer which is complete and illuminating.

I agree that having only one goal LP can work the problem out, but I already collected data by asking in the remote Nord Kivu region (DRC) the 11 DMs to pairwise compare all the alternatives (against sub-criteria), sub-criteria (against criteria) and criteria (against the goal) and therefore I would like to conclude with the AHP technique … I will keep your suggestion and maybe insert in my PhD thesis a comparison between the results from LP and AHP….thanks again!

Dear Pietro

You are most welcome, however, I don't understand your original question, if now you say that you are continuing with your original method.

Most probably I misunderstood your query

Dear M.Munier,

of course I did't explain the whole...

I am doing this research in the framework of an international aid programme in the Nord Kivu province, Democratic Republic of Congo. There, we are eliciting a decision process among local stakeholders in order to define the interventions of the next project phase. We are using a participatory approach mixing the PRA method (participatory rural appraisal) with a MCDA method (AHP). We chose AHP because of its double-ability: (1) to stimulate discussion about the differente elements composing the decision structure (goal, criteria, subcriteria and alternatives);(2) to collect data aboutdecision makers' preferences which in turn provide information to model the decision and to transform a solely qualitative discussion into a mixed method approach towards decision making.

Here is the reason why i have to continue with AHP… thank you very much for your contribution.

Dear Pietro

Thank you for the clarification. Good luck in your project!

Don't run AHP. To begin, groups don't make decisions, they have emergent behaviors, and AHP doesn't apply. The math of this is game theory. Secondly, AHP isn't mathematically valid anyway. Third, weighted sum methods have very restrictive requirements, and they are almost always improperly used. If you want to use multiattribute ranking methods, learn the theory. The weights do not represent importances, they define indifference surfaces.

Dear Pietro

I agree with what George Hazelrigg, both in his assertion that AHP is not mathematically sound, and regarding the meaning of the weights found using it.

These are trade-off values, no weights, and as that, they do not indicate importance

However, I am not sure that your problem belongs to the game theory.

If you already collected data, probably by a survey, why don't you use another method?

Dear George

I am impressed for the paper you wrote about weights

Since the early 90s, reputable MCDM experts have expressed that weights determined by AHP are not such but trade-offs, a concept that for many practitioners is difficult to grab.

Just to mention some of the scientists that have expressed their doubts about weights:

Choo, Triantaphyllou, Belton, Moskovich, Grupp, Cinelli, Department of Transport Local Government and the Region (DTLR)( (UK).

Moreover, Zardari states that chosen criteria may change their value in space and time

In addition, for years I have been postulating the same, and most especially about the fact that subjective weights are not fit to compute relative importance for criteria, and even less than suitable to be used in selecting alternatives, concept also supported by some of the above mentioned scientists.

However, yours is the first document mathematically written that demonstrates the fallacy of using these types of weights

Your paper should be read by every practitioner working in MCDM

Thank you, but actually there is nothing new in the note that I wrote. E.g., see the book, "Foundations of Multiattribute Utility," by Ali Abbas. It covers the subject fully and correctly, but perhaps in a format that is a bit longer.

AHP is deeply flawed.  The primary issue is that it relies strictly on pairwise comparison and ignores critical information about the relationship to other alternatives.  One work that you should consult is: Saari, D.G. and K.K. Sieberg. 2004. "Are partwise comparisons reliable?" Research in Engineering Design 15: 62–71.  Don Saari has written extensively on these problems and has proven important theorems about the unique benefits of the Borda Count and on how it deflects issues raised by Arrow's Theorem.There are many other problems with AHP but the most fatal is that it ignores critical information.

You might find the attached paper useful as an alternative approach.

Dear Neal

I agree with you especially when you say that 'There are many other problems with AHP but the most fatal is that it ignores critical information'

One on the many, is that AHP makes criteria pair-wise comparison a constant for each pair, when it may be not so.

In reality, the preference of criterion C1 over C2, may depend on the characteristics of a pair of alternatives considered.

For instance, comparing criteria quality and price, the DM can't say for instance that quality is more important that price and assume this assert constant. Then, it could be that comparing alternatives A1 and A2, and because the values of price and quality in each one, A1 > A2. However, it could also be that comparing A1 with A3 the DM finds that it is worth to sacrifice a little quality in A1, and getting a much better price in A3.

Therefore, in my opinion, the pair-wise comparison of criteria is flawed if the alternatives to which criteria apply are not taken into account. Of course, this is opposite to AHP procedure, which ignores reality and common sense.

Since you have 3 criteria and 8 sub-criteria why don't you try local weights (for the subcriteria) and global weights (for the main criteria) to make pairwise comparisons?

Have you classified the sub-criteria into categories (3 criteria)?

You may then obtain weights for all of them. The 3 main criteria will have weights that sum up to 1 and all 8 sub-criteria would also sum up to 1 (or 100%).

Not to mention that with the use of geometric mean you can consider multiple options from several decision makers.

AHP is not flawless but can be a very powerful tool if you can use it properly.

Dear George

AHP is a very powerful tool, for what?

Other people in RG and myself presented proofs that it is flawed, both mathematically and in procedure. I can enumerate them if you want.

Could you demonstrate that we are mistaken? You will be the first in defending this method, since nobody came across to undermine what I said in several years in this forum or in books and papers

Perhaps you can illustrate us about the utility of AHP, naturally, words are not enough, proofs and reasoning are needed.

And how would it be used improperly?

Hi Pietro, you can use Expert Choice, but it is not freeware. If you are looking for a freeware software, try Superdecisions: https://superdecisions.com/

This software includes several examples,tutorials and manuals, and should work both for group-AHP and group-ANP. Regarding the theory behind group decision-making under AHP and ANP, I suggest the following book: "Group Decision Making: Drawing Out and Reconciling Differences".

Of course, "Group Decision Making: Drawing Out and Reconciling Differences" was written by Saaty himself. Keeping in mind that AHP is not a mathematically rigorous method developed by Saaty, I would be quite careful in accepting the theory put forth in the book.

Y ear George Hazelrigg

You are absolutely right. Saaty in this book was at the same time a judge and a part.

Are you concerned with getting a valid result? If not, anything is as good as anything else. Flip a coin. If you do want a valid result, you'll need to learn decision theory. I recommend the work of Ron Howard and Ali Abbas. It's a minefield out there. Most of the literature and most of the method are not valid.

Dear George

Excellent!

Any result is valid, since we do not have a yardstick to compare a solution

Sarcastic but true. Especially your last sentence

To me, validity means consistency. If the method yields choices that are consistent with your alternatives, beliefs on outcomes and preferences, then the method is valid. If the method fails this test, it is not valid. The only way to tell the difference is to understand (comprehensively) the mathematics. You cannot rely on the work of others. Most of it is wrong.

A method that has been proven mathematically to be true based on a set of rational assumptions is valid. All others are suspect. Frequently methods are chosen on an ad hoc basis out of ignorance or because the outcome fits a preconceived notion of what should be correct. See attached for a more detailed discussion.

Dear Neal

I agree with you, especially when you say 'rational assumptions'

The paper you mention is great. I will keep it for sound logic and easy reading

Thank you

Dear Pietro De Marinis

I think this paper will be a good answer for your question.

Article Sustainable Urban Transport Development with Stakeholder Par...

Regards,

Sarbast

You need to learn decision theory. When you understand that, you won't want to run AHP at all because you will understand that it is a mathematically flawed method. The mathematics of multiple decision makers is game theory.

Thank you all, George Hazelrigg , Sarbast Moslem, Nolberto Munier , Neal Hulkower , Mubashar Shah

all your answers were very useful to deepen my understanding of decision making and of the AHP method. Indeed I am a beginner in this field of research…. what I can add to the discussion is my specific point of view coming from the fact that my approach in using AHP is quite peculiar: I used AHP as a tool for eliciting discussion in focus groups aimed at building consensus in a group of decision makers, while extracting data about their opinions and using these data not to decide but to build a "descriptive model" of the decision itself, so that THIS decision (which in the end is a participatory resource allocation problem) can be completely understood and shared by the group itself and so that it can be better explained to external people. In this perspective I found AHP a good tool… even if now I am aware of the methodological failures of the method itself…..

Dear Pietro

Very good explanation. However wouldn't it be better to use brainstorming, study each idea, and little by little to build a clear understanding of your problem? After that, an average of responses obtained, and then using some rational MCDM method?

I don't know which your scenario is, but I do know that it can't be solved by personal preferences and them assigning arbitrary weights.

When you have all the data you can build your descriptive model.

In my humble opinion AHP may be good for simple or trivial personal or corporate problems, but completely useless in complex scenarios as yours seems to be.

Just a simple question, if you need to justify a result based on preferences to the stakeholders, how do you do that? Telling them you got them by intuition?

In addition, you can't apply a hierarchical structure to problems where you not only have vertical relationships but also transversals, combined, inclusive, exclusive, conditional, etc. aspects.

Hierarchy is a copy of the military typical structure where the information and orders came from top to bottom, with no discussions. You need something more elaborated than that.

And this is the reason that time ago large companies changed their structures from being hierarchical to matricial.

Dear Pietro, I am glad to be helpful. Another approach that I applied successfully in a research project involving groups of experts/stakeholders is based on the combination of PROMETHEE II and Revised SIMOS Procedure. I suggest you to read the paper entitled "Practical guide to establishing a multi-criteria and multi-actor decision-making process: Steps and tools", authored by Aenishaenslin, Hongoh, Mareschal and Waaub. In my opinion, this is a great piece of research. Concerning the softwares for applying these methods, I used:

- Visual PROMETHEE Academic edition: http://www.promethee-gaia.net/visual-promethee.html?devicelock=desktop

- SRF: https://www.lamsade.dauphine.fr/~mayag/links.html

All the best!

For Nolberto Munier : thank you again for your suggestions. Just to answer your question, (it is not easy) I'd had to explain that my thesis is that the very action of working with such structured techniques does innovate a practical field such as the one of development cooperation where, while participation is used and spread, no scientifc support to holistic judgement skill of practitioners is found (yes, things are ruled out by intuition in the real world ...). That's why I am trying to work in this field with structured decision making techniques such as AHP (which I chose because it is the "mother" of Pairwise Ranking, one of the most spread techniques in many many development organizations' guidelines and practical manuals (such as UN-FAO, UNICEF, World Bank, Gates Foundation and many others…).

Dear Nolberto,

it is always a pleasure for me to read your comments and with your note comparing hierarchical decision structure with military operations and logic you hit the nail on the head again.

However, this is why I like AHP. In fact this is the only feature in AHP which I like. I dislike the pairwise comparisons, dislike the consistency measure, dislike the eigenvector method (I have a paper about the Pareto non-optimality of the EV calculations).

As you might know, I am dealing with passenger satisfaction and preference measures in transport systems. It is a quite complex issue, the supply quality can be described by at least 24 factors (we tried to omit some of them but that would lead to non-applicable results).

In passenger surveys, it is impossible to provide the passengers with 24 attributes without the hierarchy. They simply cannot cope with them. You mentioned the matrix structure, that might work in a multinational company but for surveys (in which the evaluators have 10 minutes approx.) this is not applicable in my humble opinion.

Since we have started using AHP in passenger surveys we are gaining good results confirmed many times by the decision makers (municipalities, transport companies, etc.).

I think our job is to make the evaluators' job as simple as possible and if necessary calculating a lot a using complex methods for analyzing the results but simultaneously with unburdening the evaluators. I agree, conventional AHP is not a perfecr tool but its improvements can be with keeping the hierarchical approach of the original method.

Dear Mubashar

Thank you very much friend.

I can say the same from all of you.

Reading the questions and answers of colleagues greatly expand my universe of knowledge.

Thank you Szabolcs

As I understand you like the hierarchy structure, I too, it is great for analyzing projects by examining its components and for design a strategy and also to model trivial problems, but for no more than that.

I am sending you a scheme of how you can describe complex and very complex projects using a hierarchical structure.

That is, you can indicate what conditions are in a complex scenario, and from their you can choose the MCDM method best suited to model it.

You are indeed in a very difficult field, in which however, and in my opinion, it is difficult, if not impossible, to model all the multiple its characteristics. For instance, in the situation that there are certain bus transfer points that only can be used by certain b us lines and not others, or you may need to work with redundancy when at certain point the passenger may choose two or more bus lines that go to the same place he wants to go.

You speak about 24 factors, which I imagine they are the same as criteria, which for me is a very reasonable. Of course you have to work with pax preferences which I guess are obtained by surveys, but remember that there are a lot of criteria that are highly related in urban transportation, such as frequency, travel time, comfort, etc. How do you introduce that in your hierarchy?

Of course, I am not telling you what to consider, since transportation is not my field, and even if it were, I wouldn't dare, but they are issues that come to my mind when writing, mainly derived from my daily bus trips in this city, and where I can watch and heard was the riders say and wish.

When you say that it is impossible to ask people with 24 criteria. I would say that it is most probable, because people don't usually have the time, the patience and the wish to answer them. I imagine that you are talking about a simple questionnaire and not with par-wise comparisons, but I don't understand why you relate that with a hierarchy. I can't understand either why you wouldn't try either with a matrix structure. For me, both have no relation with the survey. Please, illustrate ne.

In all honesty, I don't understand your two last paragraph. There is no doubt that surveys do improve service, but I am lost when you speak about the original hierarchical structure.

I am very interested in you answer. If you prefer, you can email me to: [email protected]

Dear Nolberto,

thank you so much for your notes and interest. I am sorry that I was not clear enough, maybe this paper will answer the question, why the hierarchy is necessary when it comes to transport supply quality survey for passengers: Article A dynamic analysis on public bus transport's supply quality ...

Article An AHP-ISM approach for considering public preferences in a ...

I will definitely contact you by email too, thank you again for being so helpful.

Szabolcs

Instead of any software i may suggest you develop your own calculation sheet on excel which will be more handy compare to any other. I developed my own sheet for calculation of AHP & FAHP. if you want i may share with you but it comprises of 15 parameters pairwise comprise of 15 parameters pairwise comparison in single criteria.

Dear Mohamed

Thank for your reference which I accessed, however, its mere title allows me to think that the author is ill informed, Look at it:

"MakeItRational – Analytical Hierarchy Process Software'

Since when, AHP is rational, by working with intuitions?

Where is rationality employed when the DM uses preferences?

It is difficult, at least for me, to assume that a software is rational when processing a method where that word is unknown?

I agree with Nolberto. There's no excuse for not learning some decision theory and sticking with valid mathematics.

use Super decision tools

It's always easier to solve a problem if you don't care whether the answer is correct. That's AHP.

And I would add: If you don't care that you are modeling a scenario that is approximate, in the best-case, and unrelated with the real one in others