unable to understand the difference between delphi method and modified delphi method used to develop a new tool.
Is the discussion needed during rounds between the expert members or only questionnaire feedback is enough?
Hi Dnyanesh,
There are several variants of the Delphi method, including about which kind of subjects (experts or well informed persons), number of rounds, what to do in which round, the cut off (70% or 75%), whether or not discuss in between with the experts, type of feedback (only statistics of also comments). Also, althoug Delphi can lead to consensus, e.g. about a course of action, or agreeing on a tool, it might also reveal that consensus is absolutely not possible. In that case it can be a valid outcome of a study.
The basic principle however is that each variant of Delphi approach or in each step you choose can be justified and is described such that readers understand the choices. It is better to not obscure variations, but explain them.
If the focus of your study is a new tool, the variant of design or construction Delphi applies (in contrast to predict the future e.g. for particular policies).
I found two publications extremely helpful to prepare my own Delphi:
Goodman C.M., 1987 The Delphi technique: a critique, Journal of Advanced Nursing, 12, 729-734.
and
Sackman, H. (1975). Delphi critique. Lexington, Mass., Lexington Books/ The Rand Corporation.
These explain what to their opinion are weaknesses of Delphi studies. And trying to overcome these weaknesses helped me well in choosing and justifying my own approach.
Your second question: the simple answer is: according to some variants of Delphi, the questionnaire feedback is enough.
However, if there are some unclarities based on panel feedback, these can lead to adaptation of statements in subsequent rounds to clarify.
I sometimes use the format:
Given: most reasearch nowadays is carried out by the internet, the topic of possible novelty bias can be ignored in many cases.
Question: do you think this (internet based) tool is appropriate for its purpose.
So a statement first that you do not want to discuss but the panel members to take it for granted, and then put your question or statement.
However, if you find out that from 50 statements in your Delphi study only 1 is clear to the panel, you need to intervene with e.g. a discussion (could now be online) or webinar to explain things or illustrate the tool or something.
A key feature of most studies is that the individuals completing it do fully understand it. If that is not the case, you can have intermediate discussions.
Again, this does break anonimity as well.
I have carried out some Delphi panels with health professionals focusing on requirements for electronic patient record systems. I did one round anonymous with open questions, exploring what panel members found important. Second round anonymus with statements like : the EPR shall be able to ensure patient privacy, applying ISO 27001 measures. and then 7 point or 5 point Likert scale.
For the statements that did not get consensus or where not clear we had discussion panels (waiving anonimity to some level, people seeing each other in an auditorium) but we used digital voting tools, not showing who answered. and in this auditorium, the statements were publicly discussed first, once everyone agreed with the formulation of the statement voting took place (50 persons, 9.00 - 16.00 hr with breaks, 50 statements in a day, 40 OK, 5 removed and 5 kept for small groups to further operationalize.).
So back to the beginning: see what is happening in your panel, adapt where possible your execution of the Delphi study and justify your choices.
Further suggested reading:
Couper M.R. 1984 The Delphi technique: characteristics and sequence model. Advances in Nursing Science, 7, 1, 72-77.
Goossen WTF, Epping PJMM, Dassen TWN (1997). Criteria for Nursing Information Systems as a component of the Electronic Patient Record: an International Delphi study. Computers in Nursing, 15, 6, 307-315.
Mitroff I.I., Turoff, M. (1975). Philosophical and Methodological Foundations of Delphi. In: Linstone, H.A., Turoff, M. The Delphi Method, Techniques and Applications, pp. 17-36. Reading Massachsetts, Addison-Wesley Publishing Company.
These describe 5 variants of Delphi!
Do not fall in the bias towards "recent publications". Most quality work on such methods are much older than the often misinterpreted and misused cutoff of 10 or worse 5 years back from now.
Hi, I'm currently thinking about doing a delphi research for my own thesis. I would like to warmly thank you for this answer. Researchgate suggestions may lead to great opportunity to collect some very usefull piece of information. Have a great day!
Thank you Prof Goossen, I am interested in looking into telemedicine during COVID using Delphi in the study as the main approach to gain insight , this is very helpful
Thank you Prof Goossen for your thorough and easy to understand explanations!
Hello, please take a look at this research. These statistical coefficients are used for determining the conformity or reliability of experts' evaluations, and the Kendall coefficient with a value greater than or equal to 0.7 was considered as the stopping index for the procedure of the Delphi method.
Article A roadmap for a leanness company to emerge as a true lean organization
- Badges
- Science topic
- Similar topics
- Geoscience
- Asia
- India