Brig. Gen. Paulita Cruz

The new woman general is Brig. Gen. Paulita Cruz, the current chief nurse of the armed forces. Cruz is the second active female general in the AFP after Brig. Gen. Ramona Go, who holds the distinction as the first woman general in the history of the Philippine Army. 

Cruz was the former chief nurse of the Army. She was named chief nurse of the AFP in October 2011 following the retirement of Brig. Gen. Nelia Buenaflor. Cruz finished her Bachelor of Science degree in Nursing from the Aquinas University in Legazpi City, Albay. She has also a Master of Arts in Nursing major in administration degree from the Philippine College of Health Sciences.

Cruz is a recipient of the Distinguished Service Star Award, Military Civic Action Medal and the AFP Combat Badge Award.

4 Hospitals in the Philippines with JCI Accreditation

Currently, there are 4 hospitals in the country which have achieved world-class recognition and this is simply because all four hospitals are accredited by the Joint Commission International (JCI). The accreditation is an international gold standard for hospitals.

JCI accreditation is internationally recognized as a premier credential for hospitals and other medical institutions. It is an international mark and endorsement for quality patient care and safety, and organisation management.


The Joint Commission, formerly known as the Joint Commission on Accreditation of Healthcare Organizations, and still known more commonly by the acronym, JCAHO, is an independent, not-for-profit organization that evaluates and accredits more than 15,000 healthcare organizations in the United States. It was founded in 1951.

JCAHO's Joint Commission International (JCI) was founded in the late 1990s to survey hospitals outside of the United States. JCI, which is also not-for- profit, currently accredits facilities in Asia, Europe, the Middle East, and South America. A count of JCI-accredited hospitals worldwide (as listed on the JCI website in June 2012) shows 375 hospitals in 47 countries.JCI works directly with healthcare organisations to achieve their goals of providing quality clinical care and services in safe, efficient and well-managed facilities.

JCI assesses through a rigorous on site survey process, a healthcare provider’s quality in the following key areas -

1. Access to health care
2. Health Assessment and care processes
3. Education and rights of individuals
4. Management of information and human resources
5. Safety of facility
6. Infection control
7. Collaborative integrated management
8. Facility management
9. Performance Measurement
10. Education & Rights of Patients

In the Philippines, three hospitals (St Luke Medical Center, The Medical City and the Makati Medical Center) are located in Metro Manila while Chong Hua Hospital (in Cebu) is in Central Visayas.



Article Contributor

FJ Babate is the president of the Beta Nu Delta Nursing Society

Delphi Technique

The Delphi Technique

The Delphi Technique is a method for the systematic solicitation and collation of judgments on a particular topic through a set of carefully designed sequential questionnaires interspersed with summarized information and feedback of opinions derived from earlier responses (Delbecq, Van de Ven, & Gustafson, 1975). Norman Dalkey and Olaf Helmer are generally credited with developing this research method for the Rand Corporation during the 1950s as a tool for forecasting military priorities (Murry & Hammons, 1995).

The process of Delphi consists of a series of rounds of survey questionnaires. In a traditional Delphi study, the first round consists of participants responding to a broad question, while each additional round would
build upon the responses gleaned from earlier rounds. The process is terminated when consensus is reached (Delbecq et al., 1975), and literature indicates that consensus can be reached in three rounds (Ruhland, 1993). According to Linstone and Turoff (1975), most commonly, three rounds proved sufficient to attain stability in the responses; further rounds tended to show very little change and excessive repetition were unacceptable to participants. The true Delphi technique starts with an open-ended question that is given to
participants to solicit opinions about one or several aspects of the topic (Ruhland, 1993). A modified version of Delphi dispenses with the traditional open-ended questionnaire and begins with a structured questionnaire (Murry & Hammons, 1995).


According to the literature, the Delphi technique has several advantageous features which include (1) anonymity, (2) written responses, (3) controlled feedback, and (4) statistical group responses (Dalkey, 1969; Delbecq et al., 1975). Among its advantages is the fact that participants do not have to
meet face to face, respondents may remain anonymous, domination by individuals is prevented, adequate time is provided for thinking and reflection, participants are granted flexibility in responding, and conformity issues are avoided (Linstone & Turoff, 1975; Weaver, 1988; Ruhland, 1993). According to   Sackman (1975), the Delphi method is generally fast, inexpensive, easy to understand, and versatile in the sense that it can be applied wherever expert opinion is believed to exist.

According to Sackman (1974), a disadvantage of the Delphi Technique is most Delphi experiments are probably unreliable and invalid. The literature suggests that Delphi study reliability depends greatly on the abilities of the respondent group or experts to "perform" by completing the necessary rounds and communicating accurately (Dalkey, 1969; Delbecq et al., 1975; Linstone & Turoff, 1975; Sackman, 1975 Murry & Hammons, 1995). Furthermore, the literature suggests that another disadvantage of Delphi studies arises when questions formulated by the researcher may influence the panel responses (Dalkey, 1969; Sackman, 1975 Murry & Hammons, 1995).








Mixed Methods


In the Delphi process, data analysis can involve both qualitative and quantitative data. Investigators need to deal with qualitative data if classic Delphi studies, which use open-ended questions to solicit subjects’ opinions, are conducted in the initial iteration. Subsequent iterations are to identify and hopefully achieve the desired level of consensus as well as any changes of judgments among panelists.

Statistical Treatment

The major statistics used in Delphi studies are measures of central tendency (means, median, and mode)
and level of dispersion (standard deviation and inter-quartile range) in order to present information
concerning the collective judgments of respondents (Hasson, Keeney, & McKenna, 2000). Generally, the uses of median and mode are favored. However, in some cases, as manifested by Murray and Jarman (1987), the mean is also workable.

Witkin (1984) questions the appropriateness of using the mean to measure the subjects’ responses if scales used in Delphi studies are not delineated at equal intervals. In the literature, the use of median score, based on Likert-type scale, is strongly favored (Hill & Fowles, 1975; Eckman, 1983; Jacobs, 1996).


As Jacobs (1996) states, “considering the anticipated consensus of opinion and the skewed expectation of responses as they were compiled, the median would inherently appear best suited to reflect the resultant convergence of opinion” (p.57). The use of mode is also suitable when reporting data in the Delphi process. Ludwig (1994) specifically addressed that “the Delphi process has a tendency to create convergence, and though this was usually to a single point, there was the possibility of polarization or clustering of the results around two or more points. In these instances, the mean or median could be misleading” (p. 57).

Respondents/Participants

Regarding the selection of subjects for a Delphi study, choosing the appropriate subjects is the most important step in the entire process because it directly relates to the quality of the results generated (Judd, 1972; Taylor & Judd, 1989; Jacobs, 1996). Since the Delphi technique focuses on eliciting expert opinions over a short period of time, the selection of Delphi subjects is generally dependent upon the disciplinary areas of expertise required by the specific issue.


Regarding any set standards of selecting Delphi subjects, there is, in fact, no exact criterion currently listed
in the literature concerning the selection of Delphi participants. That is, “throughout the Delphi literature, the
definition of [Delphi subjects] has remained ambiguous” (Kaplan, 1971, p. 24). Regarding the criteria used to guide the selection of Delphi subjects, individuals are considered eligible to be invited to participate in a Delphi study if they have somewhat related backgrounds and experiences concerning the target issue, are capable of contributing helpful inputs, and are willing to revise their initial or previous judgments for the purpose of reaching or attaining consensus (Pill, 1971; Oh, 1974). Helmer and Rescher (1959), Klee (1972), and Oh (1974) concur that choosing individuals who are simply knowledgeable concerning the target issue is not sufficient nor recommended. Considering the necessity of selecting the most qualified individuals, Delbecq, Van de Ven, and Gustafson (1975) specifically state that three groups of people are well qualified to be subjects of a Delphi study.

The authors recommend:

“(1) the top management decision makers who will utilize the outcomes of the Delphi study;

(2) the professional staff members together with their support team; and

(3) the respondents to the Delphi questionnaire whose judgments are being sought” (p. 85).

Delphi subjects should be highly trained and competent within the specialized area of knowledge related
to the target issue. Investigators need to closely examine and seriously consider the qualifications of Delphi subjects.

Oh (1974) indicates that choosing appropriate subjects is generally based on the judgment and discretion of the principal investigators. Jones and Twiss (1978) state that the principal investigators of a Delphi study should identify and select the most appropriate individuals through a nomination process. Ludwig (1994) also states that, “solicitation of nominations of well-known and respected individuals from the members within the target groups of experts was recommended” (p. 52). Generally, the pool of selecting possible Delphi subjects is likely to use positional leaders (Kaplan, 1971; Ludwig, 1994), to follow a review of authors of publications in the literature (Meyer, 1992; Miller, 2001), and/or to make contacts with those who
have firsthand relationships with a particular issue (Jones, 1975; Anderson & Schneider, 1993). The latter basically consists of individuals who are primary stakeholders with various interests related to the target issue or research effort.

Concerning the appropriate number of subjects to involve in a Delphi study, Delbecq, Van de Ven, and
Gustafson (1975) recommend that researchers should use the minimally sufficient number of subjects and should seek to verify the results through follow-up explorations.

Ludwig (1994) notes that the number of experts used in a Delphi study is "generally determined by the number required to constitute a representative pooling of judgments and the information processing capability of the research team” (p. 52). However, what constitutes an optimal number of subjects in a Delphi study never reaches a consensus in the literature. Delbecq, Van de Ven, and Gustafson (1975) suggest that ten to fifteen subjects could be sufficient if the background of the Delphi subjects is homogeneous. In contrast, if various reference groups are involved in a Delphi study, more subjects are anticipated to be needed. Witkin and Altschuld (1995) note that the approximate size of a Delphi panel is generally under 50, but more have been employed. Ludwig (1997) documents that, “the majority of Delphi studies have used between 15
and 20 respondents” (p. 2). In sum, the size of Delphi subjects is variable (Delbecq, Van de Ven, & Gustafson, 1975). If the sample size of a Delphi study is too small, these subjects may not be considered as having provided a representative pooling of judgments regarding the target issue. If the sample size is too large, the drawbacks inherent within the Delphi technique such as potentially low response rates and the obligation of large blocks of time by the respondents and the researcher(s) can be the result.






References


Dalkey, N. C. (1969). An experimental study of group opinion. Futures, 1 (5), 408-426.

Delbecq, A.L., Van de Ven, A.H., & Gustafson, D.H. (1975). Group techniques for program planning. Glenview, IL: Scott, Foresman, and Co.


Hasson, F., Keeney, S., & McKenna, H. (2000). Research guidelines for the Delphi survey technique. Journal of Advanced Nursing, 32 (4), 1008-1015.


Helmer, O., & Rescher, N. (1959). On the epistemology of the inexact science. Management Science, 6, 25-53.

Hsu, Chia-Chien & Sandford, Brian A. (2007). The Delphi Technique: Making Sense of Consensus. Practical Assessment Research & Evaluation, 12(10). Available Retrieved http://pareonline.net/getvn.asp?v=12&n=10

Jones, H., & Twiss, B. C. (1978). Forecasting technology for planning decision. London, UK: Macmillan Press Ltd.

Judd, R. C. (1972). Use of Delphi methods in higher education. Technological Forecasting and Social Change, 4 (2), 173-186.

Kaplan, L. M. (1971). The use of the Delphi method in organizational communication: A case study. Unpublished master’s thesis, The Ohio State University, Columbus.

Klee, A. J. (1972). The utilization of expert opinion in decision-making. AICHE Journal, 18 (6), 1107-1115.



Linstone, H.A. & Turoff, M. (1975).  The Delphi method:  Techniques and applications. Reading, MA: Addison Wesley Publishing.


Ludwig, B. (1997). Predicting the future: Have you considered using the Delphi methodology? Journal of
Extension, 35 (5), 1-4. Retrieved November 6, 2005 Retrieved http://www.joe.org/joe/1997october/tt2.html



Meyer, J. H. (1992). Rethinking the outlook of colleges whose roots have been in agriculture. Davis, CA: University of California.

Murray, W. F., & Jarman, B. O. (1987). Predicting future trends in adult fitness using the Delphi approach.
Research Quarterly for Exercise and Sport, 58 (2), 124-131.


Miller, G. (2001). The development of indicators for sustainable tourism: Results of a Delphi survey of
tourism researchers. Tourism Management, 22, 351-362.



Murry, J. W., and Hammons, J. O. (1995). Delphi: A versatile methodology for   conducting qualitative research. The Review of Higher Education, 18, 423-436.


Oh, K. H. (1974). Forecasting through hierarchical Delphi. Unpublished doctoral dissertation, The Ohio State University, Columbus.

Pill, J. (1971). The Delphi method: Substance, context, a critique and an annotated bibliography. Socio-Economic Planning Science, 5, 57-71.

Ruhland, S. K. (1993). Work force skills and competencies essential for the preparation of individuals for marketing occupations. Journal of Vocational Education Research, 18, 1-21.

Sackman, H. (1975). Delphi critique. Lexington: Lexington Books.

Taylor, R. E., & Judd, L. L. (1989). Delphi method applied to tourism. In S. Witt, & L. Moutinho, (Eds.). Tourism marketing and management handbook. New York: Prentice Hall.


Witkin, B. R. (1984). Assessing needs in educational and social programs. San Francisco, CA: Jossey-Bass Publishers.

Witkin, B. R., & Altschuld, J. W. (1995). Planning and conducting needs assessment: A practical guide. Thousand Oaks, CA: Sage Publications, Inc.

The Delphi Method

The Delphi Method was originally developed by the Rand Corporation in the early 1950s as a method for obtaining reliable expert opinion and consensus from a group in a military application (Dalkey & Helmer, 1963). By 1975, the technique/method (used interchangeably in the literature) was adapted for more than military purposes. Adaptations of the technique have been used for “budget allocations, putting together the structure of a model, exposing priorities of personal values and social goals, and for planning university campus and curriculum developments” just to name a few (Linstone & Turoff, 1975, p. 4). The Delphi method is a synthetic inquiry method. Synthetic, meaning it is not located in either its theoretical or its empirical components, but in both (Linstone & Turoff, 1975).

The Delphi Process in Curriculum Development

The Delphi method application in curriculum development typically utilizes a series of surveys (usually three rounds) to refine the opinions of experts and to achieve consensus. Round one typically consists of open-ended questions followed by two rounds of ratings-type surveys (Lee, 2006). In a Delphi study the first questionnaire/survey is in the form of openended questions. The questions are derived from the literature review and interviews with subject matter experts and stakeholders (Linstone & Turoff, 1975). This survey is open-ended in order to allow new thoughts and themes to develop that may have not emerged in the literature review and interviews, and to provide clarity. When creating a survey, the constructor needs to keep the questions and survey as brief as possible while using clear language for the target audience (Converse & Presser, 1986). Converse and Presser (1986) suggest that attention must be paid when creating a research questions such that only one topic is addressed in a question and that multiple items are not packed into one question. Multiple part questions can confuse the participant and provide
unreliable results. The use of double negatives should also be avoided in question creation (Converse & Presser, 1986).
Delphi surveys are completed independent of the collective study group. This independence curtails behavior interactions that might be counterproductive during group meetings, such as the most boisterous person controlling the debate (Arditi, 1999).

The first ratings survey in a Delphi study (round two) is derived from the responses from the open-ended questions in the first round. For reasons of clarity, the respondent is still encouraged to provide feedback for questions as to why they responded in the manner they did. At the end of the second survey, an open-ended question allows for any additional feedback or additional items that the respondent thought were missing from the survey (Linstone & Turoff, 1975).

The responses in round one are often reworded into statements for round two. Individuals in round two are asked to what extent they agree or disagree with the statement. Upon completion of round two often some descriptive statistic is used to measure the level of consensus in regards to an item. Sinead Hanafin in his
Review of Literature on the Delphi Technique (2004) found that different Delphi studies used a descriptive statistic with some threshold as the tool to identify consensus of an item in a study. Hannifin found that descriptive statistic such as percentages, standard deviations, means, medians, and ranges had all been
used in the identification of consensus in various studies (2004).

One method of defining consensus is when responses to items in the ranked survey fall in the interquartile range. Responses that fall outside the interquartile are removed from consideration (Wicklein, 1993).

Another method used for defining consensus in a Delphi study is the use of one standard deviation positive or negative from the mean score to indicate very strong consensus. Such range of scores would contain at least 68% of the scores centered around the mean (Williams & Webb, 1994). Others using ratings type surveys have defined the standard deviation cut off limit for consensus as items with a standard deviation of .75 or lower (Saranto & Leino-Kilpi, 1997).

Using the one standard deviation approach, requires a researcher to find the standard deviation of the entire survey, compare each items standard deviation with the overall standard deviation, and then eliminate items that have a larger standard deviation than that of the overall standard deviation (Lee, 2006).

The questions contained in the third survey can be ordered based on the mean rankings from the second survey. The third survey is also a ratings type survey with closed-ended questions, but with no areas for responses. The results of the third survey are analyzed and those items found not to have consensus
based on the measurement method that was pre-selected are excluded from further use. Items remaining are ranked by the mean for each question for reporting (Linstone & Turoff, 1975). The respondents for the first, second, and third rounds of the surveys should be the same study group. The study is longitudinal from round two, forward.

It is suggested that three types of expert groups be included in a Delphi method study: decision makers (those who will use the information derived from the study), the Delphi staff group (those who are designing the study), and respondents (experts whose participation and opinions are being sought) (Delbecq, et al., 1986).

It is further suggested that some common objectives of the Delphi method are to determine the range of program alternatives, to discover the assumptions driving different judgments and thoughts, to find consensus from the respondent group, to correlate the judgments and thoughts of the groups, and to share the multiple opinions among the respondents. (Delbecq et al., 1986).

The Delphi method allows a researcher to extract respondents’ opinions regarding content and skills
requirements with equal weight from each of the group’s participants (Volk, 1993). By using the Delphi survey method, precision and specificity can be obtained through the concerns identified through the multiple rounds of the Delphi survey, as previously described in this paper. The Delphi method offers the designer a process of identifying learning outcomes and narrowing the identified outcomes to a manageable list for the curriculum or class.

Given current web based technology, a Delphi study no longer has to be conducted using paper-based surveys and feedback. Computer-based Delphi applications can allow researchers to conduct data capture online, facilitating and perhaps expediting the process. This is only useful if the respondents are
capable of accessing and using such an online application and the application does not become an obstacle in the data acquisition (Adler & Ziglio, 1996).

Response rates for survey studies vary. A survey of the literature shows variation in return rates from general web-based and email-based Delphi study surveys. In a study of survey types, Schonlau, Fricker and Elliott (2002) found web-based survey return rates range from 7 to 44 percent and email surveys ranged from 6 to 68 percent. The range was attributed to variance between solicited and non-solicited surveys. Solicited surveys yield a higher return rate.

Higher quality questions also yield better return rates. Brancheau and Wetherbe (1987), in a Delphi study of Management Information Professionals, found return rates of 50 percent for the first round, 62 percent for the second round and 100 percent for the third round. Lundberg, Marshall and Helga (1998), in a modified Delphi study with a respondent formed from The Journal of the American Medical Association Editorial Board, found a return rate of 73 percent.

Validity and Reliability of the Delphi Method

Different measures of validity exist, as do many methods of inquiry systems. The Delphi method is closely aligned with the Kantian inquiry systems approach (Linstone & Turoff, 1975). The Delphi methodology uses a design structure which allows “many informed individuals in different disciplines or specialties to contribute information or judgments to a problem area which is much broader in scope than the knowledge that any one of the individuals possess” (Linstone & Turoff, 1975, p. 27). Kantian inquiry is goal-oriented -- its
target is to clarify the problem and the nature of the problem. The Delphi method provides a medium to this end, providing informed persons from many interest groups the opportunity to discuss, refine and rank the issues and problems through the multiple surveys (Linstone & Turoff, 1975).

In a review of Delphi method studies, it was found that variance reduction almost always occurs in Delphi groups between the first and the fifth rounds (of surveys), but the results are almost always known by the end of the third round.

Further rounds may impair the results (Brockhoff, 1975). The credibility of a Delphi study is grounded in its effectiveness in aiding decision-making (Hanafin, 2004). The extent to which this is the case has been subject to some examination. One systematic review of empirical studies (n= 25) comparing Delphi studies with standard interacting groups concluded, with some caution (attributed to wording of questions), that Delphi groups outperform standard interacting groups in decision making (Hanafin, 2004).

Reliability and validity are of concern in any study. Reliability and validity in a Delphi method is derived from the “number of experts, their average expertise, and the average inter-correlation of their judgments” (Hanafin, 2004, p. 41).



References



Adler, M., & Ziglio, E. (1996). Gazing into the oracle. Bristol, PA: Jessica Kingsley Publishers.

Arditi, D. (1999). The Delphi method. Retrieved September 13, 2005, from Retrieved http://www.iit.edu/~it/delphi.html

Brancheau, J., & Wetherbe, J. (1987). Key issues in information systems management. Information & Management (pp. 23-45). MN: Management Information Systems Research Center

Brockhoff, K. (1975). Evaluation: Performance of forecasting groups, In H. A. Linstone, & M. Turoff (Eds.), The Delphi method: Techniques and applications (pp. 291-321). Reading, MA: Addison- Wesley.

Hanafin, S. (2004, March 2). Review of literature on the delphi technique. Retrieved October, 2005, from http://www.nco.ie/upload_documents/ Delphi_Technique_A_Literature_Review.pdf


Lee, Y. (2006). An investigation and critique of competencies needed by human resource development (HRD) master’s degree graduates in Korea. (Doctoral dissertation, The Florida State University, 2006), Retrieved January 2, 2007, from http://etd.lib.fsu.edu/theses/available/etd-11132006-
164342

Lundberg, G., Marshall, C., & Helga, F. (1998). A Comparison of the opinions of experts and readers as to what topics a general medical journal (JAMA) should address. JAMA, 280(3), 288-290. Chicago: American MedicalAssociation.

Saranto, K. & Leino-Kilpi, H. (1997). Computer literacy in nursing: Developing the information technology syllabus in nursing education. Journal of Advanced
Nursing, 25(2), 377–385.

Schonlau, M., Fricker, R, & Elliott, M. (2002) Conducting research surveys via e-mail and the web. Santa Monica, CA: RAND

Wicklein, R. (1993). Identifying critical issues and problems in technology education using a modified-delphi technique. Journal of Technology Education, 5(1), 54-71.

Volk, K. (1993). Curriculum development using the delphi technique. Technology Teacher, 52(4), 35-40.