Dear Luis

I don't think that there is such a thing as a validation method to check results for AHP or for any other model. Since you do not know which the true result is you cannot make any comparison. In an algebraic formula you know for sure that whatever the value you put in the formula the result, even if you don't know  it, will be correct, but not in MCDM since there is not such a formula.

The only model that guaranties that you reach a true solution, if it exists, and even if you don't know it, is Linear Programming (LP), that delivers a solution that complies with Pareto optimality principle. Other than that, there is not other heuristic model that can produce an optimal result. However,r more complicate models such as those based on the genetic algorithm may produce optimal solutions.

Unfortunately, there is a drawback in LP, in the sense that it only works with one objective function and only with quantitative objectives. This is not very realistic in most projects, and this is the reason that provoked the appearance of so many heuristic models such as SAW, AHP, ANP, Promethee, Electre, Vikor, and Topsis to name the most popular.

A model based in LP such as Goal Programming can work with several objectives and delivers an optimal solution, if it exists, however it cannot cope with qualitative criteria.

In despite of that LP continues being used by tens of thousands of la large companies around the world.

Regarding the criteria to choose a model, it depends of the nature of your problem as well as its complexity. It also depends on the size and the work load needed. In my opinion, AHP is a very good model for simple problems and where criteria and independent, something that rarely occurs.

It is limited someway because there is a need of lengthy comparisons for which it is recommended up to ten criteria. The same applies to ANP which can handle criteria dependent and independent, however the load work is still worse than in AHP. For both there is good software and abundant literature since they are by far the most used models. If your problem is complicated with many alternatives and criteria I would recommend any of the other aforementioned models, which have good software.

If you problem has many criteria and alternatives, both in the hundreds, and you have restrictions in the use and availability of your resources,you need to recreate the mathematical model as close as possible as reality. In this case I recommend Simus, with also freely available software.

Hope it helps

People have blindly started to use such methods ( AHP, Boolean or others) even without understanding the inherant fabric, weaknesses, assumptions, limits, constraints ...and say for example dealing many environmental issues  which invites lots of cost and lots of time. One example say. locating proper sites for artificial ground water recharging ..how a Boolean or any other approach can provide a sound answer, unless otherwise we deeply understand such approaches and comparison of plethora of such approaches may be AHP too. A bad decision to locate a bad site will certainly results into a financial loss, environmental loss, and energy loss....What I mean to say or react with points of learned Munier and Nova , to please give some light on it. weather the Physics involved in is not important? both the inherant fabric of such mathematical approaches and the Physics of the system where these are to be employed are important,  to designate or rate the success of such approaches or suitability of such approaches , atleast while dealing with nature and physical processes involved therein.

The AHP methodology is a template built by yourself, so how can you validate it? What you can do is evaluate the “robustness” of this model or decision result, using an analysis of its sensitivity to the changes of the parameters that you used. If your model is very sensitive/susceptible, i.e. small changes in your valuations vary significantly the final decision/result/response, then you should be wary of its consistency. Otherwise, if changes you make to the input parameters/valuations do not vary your final results/decision about your options, then you can trust (validate?) your model. Normally I use the ExpertChoice, and the Sensitivity Analysis is there.

Hope it helps

Dear M.L. Gaur.

I am afraid that I don't understand what is it that you request in this paragraph:

What I mean to say or react with points of learned Munier and Nova , to please give some light on it. weather the Physics involved in is not important? both the inherant fabric of such mathematical approaches and the Physics of the system where these are to be employed are important,  to designate or rate the success of such approaches or suitability of such approaches , atleast while dealing with nature and physical processes involved therein.

1 Light on what?

2. What is Physics related to this? If you are talking for instance about environmental damage to aquifers because pesticides, I can understand, but, naturally this has to be expressed as a criterion in the initial decision matrix 

3. What is this related with mathematical approaches?

4. What is all of this related with the original question from Luis, who for me made a very clear request?

Could you explain please? I am interested in what you say

Dear Jose

You gave a very good explanation.

There is another reason regarding this issue, because you cannot validate your own work, in so doing you are playing as judge and part.

Since the weights are obtained by Eigen analysis which is fed with subjective appreciations from your own, you are not using quantitative and qualitative data that, inexact or not, pertain to a problem, such as for instance output of equipment, and qualitative data such as percentages obtained from a survey considering may be thousands of people, They are more exact that personal appreciations, with which you replace. By the way, this procedure in this last case violates ' Arrow's Incompleteness theorem', that very succinctly states  that a group cannot represent a great collective of people.

As a consequence, the discrimination of criteria values which are used for alternatives selection are not corresponding to original data but to your preferences. When you consider this fact  it is hard to believe that a sensitivity analysis done with Experts Choice really indicates robustness of the solution found.

Thanks, Nolberto, you're right.

Anyway, you are not searching a "universal regularity" with the AHP, it is just a decomposition/perception of your "mind architecture" in deciding among alternatives. So, you have to interpret the thresholds of Sensitivity Analysis just as a kind of "degrees of freedom" of your decision among alternatives, no more than this (sorry for so many quotes :)

Dear Jose

Thanks for your comments

Anyway, you are not searching a "universal regularity" with the AHP, it is just a decomposition/perception of your "mind architecture" in deciding among alternatives.

OK, I can buy it, as long you are expressing preferences that affect you or the company that you  represent because in both cases you do know how will feel the people affected by a project.

But that is not valid when the selected project will imply consequences, good or bad,  to thousands of people that you don’t know and don’t even imagine the problems they may have with the development of a certain project. To decide for them or in their behalf is, as Arrow calls it, is some sort of dictatorship, where the truth is always on your side and where it does not matter what others say or feel. In all honesty José where is the rationality of this procedure?

As an engineer, but more as a rational man, I am shocked when I read that extremely and costly decisions such as industrial locations are based on the preferences of a person or a group of persons.

I have worked many years and in several very large projects, especially in hydroelectric dams, mining and chemical plants, involving hundreds of millions of dollars each, as a planning engineer as well as project manager, in both positions, from a desk far way of the project location, but mostly in the field, in construction camps, living day and night with the construction workers and  with the people that was going to be affected by the project, seeing the consequences of measures adopted according to a company decisions. Therefore, I could appreciate the repercussion of measures taken by somebody or by a group that did not pay attention to reality. The last one, as I commented in this forum recently, provoked the halting in Canada of a hydro project with a budget of 1500 million dollars, and almost have done,, because the owner, one of the largest aluminum company in the world paid more attention to his advisors and preferred not to hear the native people complain which was affected by the project.

You have to interpret the thresholds of Sensitivity Analysis as a kind of "degrees of freedom" of your decision among alternatives, no more than this (sorry for so many quotes :)

I am sorry, but I don’t understand what you mean. Which are the thresholds? Of course the DM has the freedom between certain limits to change the weights, but on what purpose, if they are not meaningful for changing a ranking?

Remember that with said exercise the DM is trying to mimic what could happen when some parameters, out of his control, may vary.

It would appear that for me the figure of the DM is irrelevant. Quite the opposite, he is the most important component of the MCDM process and his expertise, know- how and preparedness are essential.  What I am arguing refers to timing,  i.e  the moment of the process at which his participation is paramount.

In my opinion n the DM receives documented information from the different departments of the company as well as from manufacturers, vendors, etc. and also results from surveys when there are people involved. When I say timing I mean that the DM has to check and recheck that he receives adequate and reliable information, he must select the criteria, he must follow the mathematical processing and then he gets a result.

However, I believe that it is in here where the DM opinion is vital since he has received information processed by an algorithm which is mathematically correct. However, this algorithm is normally unable to consider many issues that have nothing to do with the initial data. Is in here, at the end of the process, when he receives results based on data as reliable as possible where he must act. For instance imagine that the result says that A >B > C. Naturally, he cannot accept blindly this result, and he is right and it is his obligation to take into account his own points of view. He can say for instance ‘I don’t care what the method say, but I don’t trust this result because Contractor A has not a very good standing in the industry; he has had cases of no compliance with times, and the quality of his work is not as good as others, again, after consulting with the construction industry’, Consequently I will recommend contractor B instead of A.

In this way he is acting on a result that is as good as possible, grounded on solid basis, not on initial preferences and assumptions, which may be different from another DM. By the way, this is the manner than many high tech industries work. They design say a muck up of a new airplane  according to the best engineering and air dynamics formulas,  however, the final decision is taken after the model is tested in a wind-tunnel, that allows for corrections to be made.

Sorry for the length of my answer.

Nolberto

DearNolberto,

Sorry for delay and thanks for your so full answer!

Your point of view is very valid and such things happen many times.

And it is true that models and methodologies sometimes are misused, and abused.

I use the AHP to synthesize the preferences expressed by people that enjoy public spaces, so it is the people who are “my dictator”. If I know what people wants (evaluating each criterion) I can leave for the (good) experts the choice (and the justification) among the best alternatives that meet these criteria.

When people responses are very different, all alternatives converge and sensitivity analysis shows short gaps around best solution and other alternatives. So we must repeat and/or refine the inquiries to improve my decision model.

When the people answers are always the same, so alternatives results are quite distinct, and sensitivity analysis points large distances between the better solution and the remaining alternatives.

But of course, common sense must always prevail in the interpretation and acceptance of results.

Hope it helps, thanks again for the opportunity to think and discuss these matters.

J

Dear Jose

Thank you for your answer. This is my answer, and I hope that we can continue this interesting discussion. I am always willing to learn from others.

Jose. I use the AHP to synthesize the preferences expressed by people that enjoy public spaces, so it is the people who are “my dictator”. If I know what people wants (evaluating each criterion) I can leave for the (good) experts the choice (and the justification) among the best alternatives that meet these criteria.

Nolberto. In your response there are two words that for make are crucial but for me are non-understandable.

Synthesize

As you know it means to combine constituent elements into a unified entity. That is, if you take information from people about enjoying public spaces, do you reduce them to a single judgement?  If you have for instance a criterion that calls for people maximizing public space and considering that people opinion on that criterion for three different alternatives are, the higher the better, Alt A (Thematic park)= 30%, Alt B (Cultural Centre)= 12% and Alt C (Mall and Library)= 27 %, how do you synthesize this data? For me what people say is very clear, they express that out the three alternatives or options they prefer Alt. A. What is it that you want to synthesize, and why do you want to do that? I will appreciate your explanation because it appears involving something that I am missing.

 If for another criterion you have different attractions for each site and for a third criterion suppose you have different sports to be practiced. These three criteria are used to evaluate each alternative. What is there to synthesize?

Evaluating each criterion

May I ask you on what grounds you can say for instance that criterion C2 (Attractions) is more important than criterion C2 (Sports)?.  If you ask the people, yes, I do understand it, THEY can put a value of importance, but do you think that it is YOU as a DM, who is alien to the benefits of this undertaking, that has the capacity to evaluate which of the three criteria is more important to the people?  In all honesty Jose, I don’t feel that it is reasonable. But again, may be it is me that don’t understand the logic of your reasoning because I am failing to see something that you do. So, your explanation will be greatly appreciated

Jose. When people responses are very different, all alternatives converge and sensitivity analysis shows short gaps around best solution and other alternatives.

Nolberto.  Sorry, I don’t follow you. Why if answers are different alternatives converge? And what does converge in this context mean? That all of them have the same score or importance or those they coincide?  But if it is, then you contradict yourself by saying than people responses are different. Or did I misunderstand it?

From my point of view, it is exactly the opposite; if the answers diverge there will be a large discrimination, and then it is easier that one alternative sticks out from the others. The problem is when they have very close scores, even equal, and then how does the DM select?

I don’t understand why you introduce sensitivity here. You say: ‘Sensitivity analysis shows short gaps around best solution and other alternatives’

Could you explain your assert,  because I don’t see any relationship between sensitivity analysis and discrimination

Jose. So we must repeat and/or refine the inquiries to improve my decision model. When the people answers are always the same, so alternatives results are quite distinct, and sensitivity analysis points large distances between the better solution and the remaining alternatives.

Nolberto. Naturally, if you repeat or refine your inquires you can improve your model, but if answers are always the same I don’t see how you can improve it. I am sorry but you lost me with your last sentence. Would you please clarify it?. Again, why sensitivity analysis will point large differences? due to what? And again, sensitivity analysis is for determining how the input varies when you change certain parameters that are inputs. I don’t see any relation with what you say. Most possible I am missing something that you said.

Jose. But of course, common sense must always prevail in the interpretation and acceptance of results. Hope it helps, thanks again for the opportunity to think and discuss these matters.

Nolberto Precisely, that is my point, because common sense tells that you cannot ignore or override with your preferences what the majority say.

And also consider that unfortunately, because by preferences that may include VESTED INTERESTS a DM may select an alternative which is not the best for the company, although profitable for his wallet. Does it sound familiar?

Nolberto

Dear dr. Munier

Thanks for the wonderful body of knowledge contribution about AHP result validation.

Please guide me regarding the academic literature (journals, book chapter, conference, patents) stating there is no formal method to validate the results of AHP and suggesting the ones discussed by you. As i have to refer to your content or any other stating the issue.

Thanks

Dear Umara

Thank you for your request but I am afraid that I can't do that.

You have probably to look in Scopus the literature you are looking for, and it is a very large effort, according to the thousands of papers written on AHP and its applications.

I don't have any suggestion simply because it does not matter the |MCDM method, there is no way to validate it, simply because you don't know which the 'true' results is,therefore, you have noting to compare to