There are various ways to go with. I think selecting the criteria and having the data available to measure those criteria is the first and a very influential step in an AHP

Which criteria you choose is decisive for the outcome of the MCE. It usually is literature backed. If there is no literature, back them up on a solid basis.

Concerning the levels: I would say, the less levels you choose, the better. However, the more criteria, in the same branch of the decision tree, the more comparisons you need to make. I would say more than 3-4 criteria on the same level makes the questionnaire too long.

Chapter 2 of the following (yet unpublished) paper might help you. It introduces a novel way to standardize the input values ("Aggregating Value Functions: A Parameter-free, Uncertainty-aware Method to Elicit and Aggregate Value Functions from Multiple Experts in Multi Criteria Evaluation")

http://josis.org/index.php/josis/article/view/368

hi,

1. In order to collect Data for solving multi criteria problem using AHP, you should use the questionnaire; and these questionnaires should be answer by the managers or experts of the selected case.

2. the used criteria for your project or study must be reasonable and proper. for this purpose you can used the related criteria of your case by adapt on the related literature.  if you want proposed a new criteria set, your criteria must be reasonable and proper and i advice you which consult with your professors.

For data collection you should visit.in the case company. The purchasing department will provide to valid data for AHP analysis.

Arvind

There is not doubt that the DM needs data for establishing his preferences.

That applies nicely  for instance, for a company contracting new personnel

One of the criteria could be for example 'quality level' (according company standards), or 'lowest performance accepted' (tolerance to company needs), or 'level of expertise' (according to complexity of a work position).

In this case the DM knows what he or his company wants, and also he needs data. This data may come in the form of presentations made by potential candidates. Then studying potential candidates background the DM can make sound decisions.

This is the way used by entreprenerurs, headhunters and Human Resouces Departments to contract new personnel, although probably they don't use much MCDM methods

However, if in the case of purchasing equipment for instance, the DM gets from the purchaqsing or engineering department firm values for say three different equipments, which is the function of the DM?

Is he going to modify firm values according to his experience?

That is, what can the DM say about the criterion 'Cost of equipment'? Is he going to correct homologated values from suppliers because his preferences?

For another criterion such as 'Performance of equipment from other users', the DM will also may have information perhaps not quantitative but linguistics, as 'very good', 'reliable', 'por performance' and so on. He can translate those values into cardinals by different methods, but where are his preferences?

I would like very much if somebody could explain me this, because I never undrestood the role of the DM when there are quantitative values, nor when he is addressing aspects that he CAN'T have complete information, that is involving a large quantity of people.

How can he decide for thousands?

In addition, and as other colleagues suggest, the selection can be based on 'related literature', I am afraid that I disagree, because each project is unique, each project has its own demands and restrictions, and projects can involve different fields such as Medicine, Geology, Environment, Social issues, etc, at the same time, nor shared by others

In my modest opinjon practitioners must study  their projects very consciously and then select the criteria according to them. Of course, he can extract information from other SIMILAR projects, but this is only illustrative.

The DM has to créate his own criteria considering posible and even unforfeseen aspects of the projects, no matter if he needs 5, 10 or 78 criteria for that.

Dear Syed I want to back the excellent and very well argumented answer posted by Norberto, some of their comments could seem to oppose the previous one but I will prefer to point in ther simmilarities in order to enhance your understanding (or perhaps get you really confused!).

Beni pointed about the importance (as the very first step) of take care about the criteria to use in order to make a good decision, then the number of criteria and levels are of secondary importance and the unique compromise to decision makers is to build a suitable model of the particular problem that they are dealing with. In any case the compromise is a natural consequence of the dilemma regarding simplicity versus reality that is linked to any modelling endeavour.

In AHP Saaty always remembered us to be in line with Miller's criteria of Magic Number Seven plus or minus Two, and he advised stronlgy to break down the number of alternatives by means of some criteria (or the suitable factor in the inmediate hierarchy level) in order to work with manageable matrix size and imposing not high-demanding cognitive loads to experts consulted.

Another very interesting point in Norberto post is regarding the divide between qualitative and quantitative data to be considered by decision-makers. I agree that multicriteria decision making methods that relies on expert opinion are more suitable and useful when data are qualitative or provided in a noisy environment (i.e. surrounded by ambiguity and/or uncertainty).

Syed, in terms of the substantial part of your question regarding the data collection when you need to elicitate judgments from others or yourself, Armin answer is absolutely clear: you will need to design and apply some questionnaire in order to get from  experts (or in some ocassions from crowds) the wisdom to effectively rank your options and make a decisions. Questionnaire design suitable to AHP applications have some constraints better illustrated by means of examples. I use survey monkey (a commercial web survey service) in my research to get judgments from experts or crowds (data in any MCDM application) and you can see the examples following the hyperlink in this post.

Welcome to the MCDM user community and do not hesitate in questionning if you need further information about the issue.

Manuel

https://es.surveymonkey.com/r/SAMPLE-AHP

Dear Manuel

I assume that when you speak about second literature you are referring to papers published in journals.

I am afraid that I don’t concur with your definition of CR regarding alternatives and criteria.

In my opinion CR refers to the lack of consistency in DMs preferences. In reality it is like a licence to be wrong, since if the first DM appraisal shows a high CR, just let’s modify it! It means that he must change his first evaluation according to what algebra says. Really beautiful!!

I agree with you regarding that a high CR may show lack of expertise or experience

Could you please elaborate on the subtle difference on MCDM as a research tool and as a decision-making tool?

I am not sure I understand when you say that we suppose that population have enough and qualified knowledge of the issue. Which issues? The technical aspects of a project?

Of course, it is not expected.

If you refer to the issue about how projects can benefit or damage them, yes, I agree.

Conducting a survey means extracting information from people about what they think and feel regarding each project, that is, in which way each of them will benefit of make their life more difficult.

For instance if a project calling for building an urban highway has two options, namely a) at street level or b) elevated, people can say that they don’t like, amongst other things that option, because it will partition the city, and then it will alter their life because schools, supermarkets, hospitals, community centers, etc. will be on the other side of the highway, and the question is how are they going to reach them?

Another real example using the same highway shows that the DMs didn’t consider that rainwater coming from the mountains to the sea, will be stopped by the street level high and then flooding  the area. Do you know who told the DM about this?, a little lady living in the area that went to the project office and ask to speak with the people in charge of the project…….

The defenders of the street option can argue that the elevated version will be an eyesore for the city, as it really happened in San Francisco, until an earthquake toppled it down and many people where happier.

In my opinion, this is the kind of information that needs to be gathered, statistically ordered and they fed to the decision matrix. No DM  can produce this type of synthetic information that can involve thousands of people.

 Bye the way I tried to open the file you mentioned and it is asking for a password

Dear

you have to collect date by many different way such like questionnaire, interview, company website,

Then select the criteria

and finally using AHP

Dear

There are many alternatives to collect the data:

1-To identify the critical selection parameters (according literature review or an expert survey)

2-To assign importance values (using an AHP software or directly from a previous survey). The advantage of using an AHP software such as Expert Choice is the verification of transitivity (that measures the degree of inconsistency of the values)

3- Finally, you need to create a technical performance matrix for each alternative and  use the previous values of importance in a matrix multiplication.

I strongly recommend the use of an AHP software.

Regards,

Jaime 

Dear Jaime

You said that the 'The advantage of using an AHP software such as Expert Choice is the verification of transitivity (that measures the degree of inconsistency of the values)'

And this is true. However, the fact that the DM finds a consistency or not, might not be related with reality.

Suppose that there are true values for instance on three persons regarding their IQ which are not consistent, for instance A=118, B=112 and C= 120. It can be seen that A>B, B