I teach a course on qualitative and quantitative research methods for graduate students. I would like to add something for Delphi method. Any recommendations would be appreciated.
Erik
https://www.investopedia.com/terms/d/delphi-method.asp
https://www.projectsmart.co.uk/delphi-technique-a-step-by-step-guide.php
Delphi was invented by Norm Dalkey at the RAND Corporation during the Cold War to provide a quantitative method for predicting the future ("When will country X have Y capability?"). Before Delphi, these predictions were mainly qualitative, the product of group thinking; and the loudest voice often won the day (and was often wrong). Here is a the original Delphi article (1966) plus critiques of the method and a number of specific cases.
https://www.rand.org/pardee/pubs/futures_method/delphi.html
I've done a number of Delphi projects and found the results to be worth the effort, Easy and low cost to execute, if you have access to a panel of experts. Strongly advise using anonymous feedback in each round, so respondents' reputations don't influence subsequent estimates. At the end it is always interesting to see where different respondents landed... often there are clear "schools of thought," and it is interesting to follow up and ask those with different views "what assumptions did you make?"
Excellent find, Yang Wang!
In my experience, there are two very different situations in which Delphi methods are uniquely valuable. The first is the original purpose of the Delphi method: to help a group of experts reach consensus about important variables (e.g., estimates of an adversary’s capabilities) or to help them make predictions about the future (e.g., in what year will electric vehicle sales exceed sales of cars with internal combustion engines?). The predictive uses are particularly interesting, because one can return later to see how accurate the experts’ predictions were. We don’t have very many of these post-hoc analyses of predictive Delphi studies – how accurate was the group overall, which experts were most accurate?
The second use of the Delphi method is to reduce the variance in the opinions of an expert panel and then discover clusters of respondents who hold different views or assumptions. We might call these “schools of thought” about the topic. It is much easier to make progress toward a shared view if one has two or three schools of thought instead of a large population of experts competing for influence. It is easier, when one has only a few schools of thought, to form and test well-defined hypotheses that reveal who is right.
In both of these use cases, and in the many “how to do it” articles, there is a clear message: it is important to have a large, diverse, well-informed panel of respondents. One of the concerns in recruiting a panel is “does this person have enough expertise to make an informed judgement? Can they contribute valid information, or will they merely add noise to the data?” My advice would be, “don’t worry too much about this issue.” That is because the people who are least informed about a particular topic are the least set in their opinions, and they are easily persuaded by others after the first round of polling. This tendency to shift toward the group’s opinion gives the best informed experts’ opinions the most weight after a few rounds (traditionally four). That self-weighting aspect of the Delphi method is part of the magic of the approach.
Thank you for your encouragement, Dr. Bridge.
From my personal perspective, the paradigm which drives the Delphi method may be critical. At present, many Chinese health service researchers like to use constructivism to drive the Delphi method, trying to construct a series of indicators, but it requires subsequent practice tests and revisions. Another approach is to use the pragmatism to drive the Delphi method, which aims to form an expert consensus and to achieve prioritization of some specific issues, but it needs to be combined with the implementation science to ensure that its finding can be conducted and perfected in practice.
Hello, please take a look at this research. These statistical coefficients are used for determining the conformity or reliability of experts' evaluations, and the Kendall coefficient with a value greater than or equal to 0.7 was considered as the stopping index for the procedure of the Delphi method.
Article A roadmap for a leanness company to emerge as a true lean organization
- Badges
- Science topic
- Similar topics
- Thesis Research