Real-time Delphi (RTD) is an advanced form of the Delphi method. The advanced method "is a consultative process that uses computer technology" to increase efficiency of the Delphi process, that is an interactive survey in which the participants are provided with anonymised feedback on the experts' forecasts from the other participants previous interactions with the survey as well as the reasons they provided for their judgments.
Definition and idea Gordon and Pease define the advanced approach as an innovative way to conduct Delphi studies that do not involve sequential "rounds" and consequently lead to a higher degree of efficiency with regard to the time frame needed to perform such studies. Friedewald, von Oertzen, and Cuhls underline that aspect by writing, in "a Real-Time-Delphi, the participants do not only judge twice but can change their opinion as often as they like when they see the aggregated results of the other participants". So, here it becomes clear that the Real-Time Delphi approach requires real-time calculation and provision of group responses. Friedewald et al. further state that the Real-Time Delphi method has beneath its explorative and predictive elements also normative and communicative elements. These latter are investigated by Bolognini, who explores the potential of computer-based Delphi as a communication technique for electronic democracy. Comparative studies of von der Gracht and colleagues have revealed that Real-time Delphi studies are comparable to the outcome of conventional round-based Delphi surveys.
History The basic idea of a real-time, therefore computer-based (usually web-based), Delphi approach originates in a paper published by Turoff back in 1972 about an online Delphi conference conducted in the United States. The conference was characterized by remote locations of participants, an online tool to access and give judgments, anonymity of the participants, continuous operations and analysis of results (i.e. participants were able to see given answers of the other participants in real-time), as well as asynchronous participation (i.e. participants could independently login and logout as often and when they desired). The stated aspects are some of the key characteristics of Real-Time Delphi studies, which shows that the original idea of conducting such studies can be traced back to the respective year. Today, nevertheless, technological innovations and advanced computer aided design possibilities (e.g. high-speed internet connections, high definition graphic, and advanced processor performance) facilitate more sophisticated studies in this context. The general idea to develop a faster advanced form of Delphi studies by using ideas and basic concepts of Turoff, was initiated by the U.S. Defense Advanced Research Projects Agency (DARPA), which awarded a grant in 2004 to develop an approach to improve "speed and efficiency of collecting judgments in tactical situations". A small software company named Articulate Software in San Francisco was awarded an innovation research grant to develop what DARPA was asking for. Adam Pease, principal consultant and CEO of Articulate Software, published the findings and methodology together with Theodore Gordon in 2006. Based on the findings in this seminal paper, Heiko von der Gracht developed a second Real-time Delphi platform until 2008. In subsequent years, von der Gracht and colleagues used the platform extensively for prospective research studies. In 2011, von der Gracht and Gordon presented a joint paper at WFS Annual Summit 2011, which summarizes the lessons learned of 40 RTD studies across the two platforms. In 2017, Angenheyster and colleagues published a study, which compares various Real-time Delphi platforms. Analysis criteria include among others visual appearance, data output, user friendliness, administration, and types of questions. A cross-disciplinary study by Beiderbeck et al. focused on new directions and advancements of the Delphi method, including real-time formats. The authors provide a methodological toolbox for designing Delphi surveys including among others sentiment analyses of the field of psychology.
Differences between conventional and real-time Delphi method The question arises how a Real-Time Delphi study differs from a Conventional Delphi study. The basic framework is to think of a Delphi study which is conducted in the form of an online questionnaire. However, a Conventional round-based Delphi study conducted via the internet is called "Internet Delphi". The basic difference to Internet Delphi is that the process of a Real-Time Delphi is not characterised by single iterated rounds. In fact, real-time calculation and provision of responses are the key characteristics of Real-Time Delphis. Various other labels for Real-Time Delphi can be found in literature and many authors are not completely aware of the differences: "Electronic Delphi", "Computer Delphi", "Computer-aided Delphi", and "Technology Delphi". However, it is important to truly understand the design and process a researcher has chosen to find out whether real-time calculations and provisions have been applied or not. The typical Real-Time Delphi process can be described in the way that participants get access to an online questionnaire portal for a certain time frame, within which they are allowed to log in and log out as often as they want. Whenever they login, they will see all their quantitative and qualitative answers of previous sessions and they can change all answers as desired within the given period of time. Besides their own answers they will see the ongoing – hence, real-time – responses of other participants, and with regard to metric assessments the group as a whole will be visualised in terms of median, average, and interquartile range (IQR). In modern RTD platforms, the group's distribution is conveniently presented visually in the form of a histogram, sometimes overlay on the rating slider. It has to be pointed out that the numerical visualisations as well as the qualitative inputs change in the course of other participants changing their responses. Consequently, a participant can find out to what extent his own responses from an earlier point of time are still within the group opinion (i.e. IQR). The core innovation, then, of Real-Time Delphi studies is the real-time calculation and provision of results.
… excerpt ends here. Continue reading the full article.
