I have responses from Experts in AHP ranking 13 variables. I wish to compare the agreelibiality and disagreement better experts in the same cluster e.g Engineers,Geologists etc. How different were the responses of Engineer and Engineer 2 etc
How do I go about it?
Thank you Theo,
Instead of getting the average of an engineer, reason being being if there was an outlier response it will be spilled over to the average and consequently to my comparisons, how would I do comparisons independent of an average?
Dear Petter,
Firstly, why you are concern about the disagreement. Measuring disagreement is not the goal for a project where AHP is applied. Rather, aggregation of different preferences is expected. If you want to minimize the strong disagreement, that is, if you want to reduce disparity, then, you can use the geometric mean of the preferences or weigh/principal eigen values. Secondly, if there is any significant outlier is found, then, it will be checked by consistency ratio.
If the measurements are on a numerical scale, you can use the correlation coefficient between two groups (e.g. between engineers and geologists). If the responses are not numeric or this is otherwise not appropriate to what you're doing, you can use the Kappa statistic. Basically Kappa is the (observed minus expected agreement) divided by (1 minus expected agreement), or (O - E)/(1 - E). There will be some agreement between the two groups just by chance, so conceptually, Kappa is the amount of agreement beyond chance divided by the amount of possible agreement beyond chance. A statistics textbook will explain this computation (or possibly you can find it online, but be careful of your source).
Another possibility, depending on your data and the nature of your investigation, is to use sensitivity and specificity. This can be used if one of the groups is used as a "gold standard", i.e., one group is the "experts" and you want to see how the other group compares to their expert judgement.
There is also the McNemar test for comparing paired proportions, but this is usually used for before-and-after studies or repeated measurements on a binary variable. From what you say, it sounds like the Kappa statistic is what you want. The Kappa value may range from less than zero up to 1. Usually, anything above 0.60 is considered good agreement. Zero is no agreement at all, and less than zero means there is less agreement than what would be expected by chance alone.
Dear, Kazi Masel Ullah,
I accomplished a project using AHP where i had many groups of experts giving opinions based on AHP methodology.
That not withstanding i want to analyze if really it was necessary for me to go the different groups of experts or it would be enough to ask any professional person irrespective if he is related to my work or not. I want to know how the engineers compared amongst themselves, same to Geologists, Surveyors etc.
Afterwards analyze how engineers compared to geologist or Surveyors compared with administrators etc.
Dear Jerry,
Thank you for your response , my data is numerical, i will be exploring correlation coefficient and the Kappa statistic
Read this article, can be helpful
Attributes Weights Determining Peculiarities in Multiple Attribute Decision Making
Methods
internet.ktu.lt/lt/.../1392-2758-2010-21-1-32.pdf
Dear Peter Mwangi Macharia;
I can not understand your question very well. "agreelibiality and disagreement better experts in the same cluster " "the better experts in the same clusters"
You try to model a problem based group decision making principles as far as I understand.
You also try to model your problem by the Analytic Hierarchy Process (AHP) as far as I understand. .
The father of the method is Prof. Dr. Thomas L. Saaty. He is an American mathematician, who was born in 1926 in Mosul, Iraq.
I can not give a clear reply to you because I do not understand your question very well. However, I recommend you to read the following documents and visit the webpages:
Likert, R. (1932). A Technique For The Measurement Of Attitudes. Archives Of Psychology, No: 140, New York, USA.
Miller, G.A., (1956). The magical number seven, plus or minus two: some limits on our capacity for processing information. The Psychological Review, 63, 81–97.
PITT KATZ (University of Pittsburgh, Joseph M. Katz Graduate School of Business and College of Business Administration, Business Analytics and Operations). http://www.business.pitt.edu/katz/faculty/saaty.php
Saaty, T.L. (1980). The Analytic Hierarchy Process. New York, USA: McGraw-Hill.
Saaty, T.L. (1987). The Analytic Hierarchy Process-What It Is And How It Is Used. Mathematical Modelling, 9(3–5), 161–176.
Saaty, T.L. (1990). How to make a decision: The Analytic Hierarchy Process. European Journal of Operational Research, 48, 9–26.
Saaty, T.L. (2008). Decision making with the analytic hierarchy process. International Journal of Services Sciences, 1(1), 83–98.
Saaty, T.L. (2010). The Eigenvector In Lay Language. International Journal of the Analytic Hierarchy Process, 2(2), 163–169.
Saaty, T.L. (2011). The Serious Omission Of Comparisons In Aristotle’s Laws Of Thought. International Journal of the Analytic Hierarchy Process, 3(1), 180–181.
International Journal of the Analytic Hierarchy Process http://www.ijahp.org/index.php/IJAHP
http://www.superdecisions.com/
http://expertchoice.com/
http://en.wikipedia.org/wiki/Analytic_hierarchy_process
http://en.wikipedia.org/wiki/Thomas_L._Saaty
http://en.wikipedia.org/wiki/Logical_Decisions
http://en.wikipedia.org/wiki/Criterium_DecisionPlus
http://en.wikipedia.org/wiki/Decision_Lens
http://en.wikipedia.org/wiki/Analytic_hierarchy_process_%E2%80%94_Leader_example
If you need I can also send the list of more documents and papers via e-mail.
If I can understand your problem and model better, I can give more detailed reply (if AHP: Analytic Hierarchy Process).
Good luck with your research.
Have a nice day
Best Regards
Burak Omer Saracoglu
- Badges
- Science topic
- Similar topics
- Mathematical Sciences
- Statistics
- Statistical Software
- SPSS