Identification and Prioritization of Patient Experience Evaluation Criteria of Healthcare Using a Hybrid Method Delphi and Fuzzy Analytic Hierarchy Process

Introduction

Policymakers and administrators are interested in evaluating hospital processes to enhance decision-making strategies and tools (1). These stakeholders aim to gauge their ability to meet patient expectations and needs. Assessing patient experience of healthcare is an essential criterion in this endeavor, aiding to determine patient preferences and individual requirements (2, 3). Given patients' central role as primary healthcare customers, integrating their experiences into care processes and outcomes improves the quality of services.

Transitioning from treatment-centric to patient-centric healthcare underscores the need to evaluate and prioritize patient experiences of healthcare throughout treatment, aligning with ethical principles (4-6). The researches have mainly focused on identifying factors affecting the patient's experience of healthcare. These factors include communication with health care providers, family participation in decision-making, emotional support, physical environment, discharge methods, respect for dignity and privacy, and information provision (7-9). Positive perceptions of this process substantially impact patient loyalty, confidence in clinical outcomes, and readmission rates (8). Previous studies showed that evaluating patient experience of healthcare is pivotal for enhancing care, informed decision-making, meeting patient expectations, and efficient healthcare performance management (10). However, there are challenges in accurately and comprehensively measuring the patient experience. The relative importance of these criteria can vary depending on the context and perspective of different stakeholders. Therefore, there is a need for a comprehensive and structured approach to identify and prioritize these criteria. This framework serves as the basis for assessing the relative priority of criteria influencing a patient's healthcare experience regarding both one another and other healthcare outcomes (11, 12).

There are various methodologies for identifying and prioritizing these criteria. One approach involves integrating the Delphi technique with a multi-criteria decision-making method, such as the Fuzzy Analytic Hierarchy Process (FAHP). Typically, during the initial phases of research, Delphi-based expertise-driven approaches are utilized for criterion identification. Subsequently, the AHP or FAHP techniques are employed to assign weights to the identified criteria and construct the necessary decision-making model. This hybrid approach seeks to quantitatively capture the value judgments derived from group decision-making processes, thereby enhancing the precision and reliability of this process (13-18). Consequently, this proposed approach may be introduced as a structured guideline. Its combined application empowers healthcare managers and decision-makers to enhance health and medical services, facilitating more informed decisions in delivering quality care to patients. This approach synergizes expert knowledge and experience with mathematical methods to inform decision-making.

Since Mashhad city is a prominent hub for health tourism in the northeastern region of the country, it is imperative for healthcare service providers in the city to elevate the quality of services. This is vital for attracting health tourists, retaining existing clientele, and drawing new visitors. Hence, there is an imperative need to design a model in order to evaluate the patient experience of healthcare. Such a model can guide improvement efforts and address existing challenges. Therefore, the present study was conducted with aim to identify and prioritize patient experience evaluation criteria of healthcare through a hybrid approach involving the Delphi method and the Fuzzy Analytic Hierarchy Process.

Methods

This mixed-method research was conducted in the teaching hospitals affiliated with Mashhad University of Medical Sciences (MUMS) in 2022 and comprised two distinct stages (qualitative and quantitative study, respectively).

The first stage consisted of four key steps. Step 1 involved the development of a framework for evaluating patient experiences of healthcare. This framework was crafted based on a comprehensive review of existing literature related to patient experiences, evaluation priorities, and various patient evaluation models across the globe during 2001 to 2022. The research team conducted extensive searches on institutional websites and electronic databases, including Google, Google Scholar, Scopus, PubMed, Magiran, and ISI, utilizing keywords such as evaluation, patient experience, patient-centered care, patient satisfaction, and patient responsiveness. Persian or English published articles with aim to introduce diverse tools for assessing patient experiences of healthcare were selected. Studies with inaccessible full texts were excluded. A total of 580 studies were initially identified, and based on their relevance, 127 articles were remained. After reviewing their abstract, 31 studies, which specifically focused on patterns for evaluating patient experiences, were included in the study.

Step 2 involved the creation of a comparative matrix for analyzing and distinguishing the various patterns identified in the selected studies. This analysis led to the extraction of a comprehensive framework.

Step 3 focused on ensuring the reliability of the framework. The agreement coefficient, specifically the kappa coefficient, was utilized to assess the consistency of the framework. Two independent researchers analyzed the models using the framework, and any discrepancies were resolved through consultation with a third researcher.

In Step 4, a questionnaire was designed to validate and achieve consensus on the final framework. The Delphi process was employed because presenting the results of studies based on group decisions typically involves a limited number of experts (20, 21). In this regard, 50 experts specializing in healthcare quality were purposefully selected, with 20 individuals ultimately participating in the study. The Delphi method, initially introduced by Dalkey et al. in 1969, is a widely recognized approach for facilitating group decision-making in various fields (19). The importance of using this collaborative approach is developing a set of indicators by involving different groups of experienced stakeholders with the possibility of revising their judgment in different rounds Delphi until reaching a consensus (20).   The success of this approach is based on the principle of consensus, so the cutoff point in this study was considered 75% (21). This method has been described in detail in various studies (21-23).

The experts selected for the Delphi process included hospital managers, matrons, and the head of the hospital quality improvement unit from Mashhad University of Medical Sciences. They all had at least 5 years of experience working in hospitals (Table 1). The experts were provided with a questionnaire containing five response options ranging from "completely agree" to "completely disagree." The questionnaires were administered in person or sent via email along with an official letter, ensuring the confidentiality of their information and obtaining their consent.

Table 1: Sociodemographic information of the panel of experts

In the second stage of the study, the researchers developed a questionnaire to collect data for the pairwise comparison phase. The questionnaire consisted of two parts: sociodemographic variables (such as gender, education level, managerial position, and work experience) and patient experience evaluation criteria of healthcare (12 criteria and 32 sub-criteria) (Figure1). According to the characteristics of the panel of experts in approach FAHP (22, 23), thirty experts in the field of quality management of hospitals including the director, matron and the person in charge of the hospital quality improvement unit of Mashhad University of Medical Sciences with at least 5 years of experience completed the questionnaires. The data collected were input into Microsoft Excel and analyzed using the FAHP process. The FAHP methodology, tailored to the characteristics of the expert panel, was employed to determine the relative significance and weights of the evaluation criteria.

Chang's Fuzzy Analytic Hierarchy Process (FAHP) is a method based on fuzzy set theory that facilitates option selection and problem-solving. This methodology has been elaborated in the previous studies (24-26).

Figure 1. A hierarchical structure of the criteria of patient experience evaluation of healthcare

The set of objects, X = {x1, x2, . . . xn}, is juxtaposed with the target set, U = {u1, u2, ...um}. Chang's extension method guides the analysis for each specific target, denoted as "gi." This procedure entails an extension study for each individual member of the set, exemplified below (where Mygi, with y = 1, 2, . . ., m and i = 1, 2, ...., n, represents a triangular fuzzy number). The sequence of the steps is as follows:

Step 1. Determining the ranking of decision-relevant criteria (Figure1).

Step 2. Definition of fuzzy numbers for pairwise comparisons based on the principles of previous literature (27, 28).

Step 3. Constructing a Pairwise Comparison Matrix

In this matrix, the entries are characterized by fuzzy numbers, where 'j' and 'I' denote the column and row indices of the alternatives, respectively. In scenarios encompassing multiple decision-makers, each matrix entry adopts the form of a triangular number, its vertices signifying the minimum, mean, and maximum values, as gleaned from the administered questionnaires.

Step 4. Calculating  for each row of the pairwise comparison matrix using equation 1:

The value of is calculated by applying the fuzzy summation law for a specific matrix so that:

The value of  first will be counted using the fuzzy addition law for .

Then, the inverse vector is calculated as follows:

Step 5. The degrees of possibility are calculated. After that, the degree of probability of a convex fuzzy number is more significant than k convex fuzzy numbers Mi (i=1,2,3....,k) by equation (5).

Step 6. For j = 1,2,...,n with j ≠ i, d' (Ai) = min V(SI ≥ Sj), .normalized vectors are obtained through normalization by equation (6):

Step 7. Finally, non-phase normalization "W" is calculated as follows:

All calculations are carried out in Excel software.

Ethical Consideration

The present study was approved by the Vice Chancellor of Research Affairs of Mashhad University of Medical Sciences and  the Ethics Committee of Mashhad University of Medical Sciences (ethical code: IR.MUMS.FHMPM.REC.1400.118).

Results

Following a comprehensive review of the literature, 12 distinct patient experience evaluation models were identified, each characterized by its unique set of criteria. These models are as follows: HCAHPS (8 criteria), QPP (10 criteria), PPE-15 (7 criteria), NHSIP (8 criteria), SIPES (6 criteria), HKIEQ (9 criteria), NORPEQ (6 criteria), PPQ (5 criteria), PEQ (10 criteria), I-PAHC (5 criteria), PREM-CCH (6 criteria), and CEPQ (4 criteria).

The criteria examined within these patterns were individually documented for each model (Table 2). Furthermore, the study meticulously scrutinized the instances of overlap between criteria across these various patterns. The findings of this analysis revealed that certain criteria appeared with higher frequency and repetition. Specifically, criteria related to the physical environment, provision of information to patients, the manner in which treatment services are delivered, and communication practices with the treatment staff emerged as the most frequently recurring criteria. Conversely, the criteria related to pain management, patient acceptance, respect for patient dignity and privacy, discharge procedures, and patient, family, and relative participation were among the less frequently encountered criteria (Table 2).

Table 2: Criteria examined in patient experience evaluation frameworks

)communication with the nurse (C1), Communication with the physician (C2), Providing nursing services (C3), Providing physician services (C4), Providing information to the Patient (C5), discharge from the hospital (C6), Physical environment and hoteling (C7), emotional support (C8), Participation of family and relatives (C9), Respect for the dignity and privacy of the Patient (C10), pain management (C11), Access to needed medicines (C12), Patient participation and patient-centered care (C13), How to be admitted to hospital (C14), Professional competence (C15), How to manage hospital costs (C16), Efficiency (C17), Health consequences (C18)(

Figure 2. Prioritizing criteria related to patient experience evaluation

The frameworks extracted were classified separately by two researchers, and the agreement coefficient of the parties was calculated as 0.84. The initial framework in 14 criteria and 57 sub-criteria was designed as a questionnaire scored in a 5-option Likert scale to be used in the Delphi approach. In the first round of Delphi, 2 criteria and 25 sub-criteria scored less than 75% and were eliminated, and the rest entered the second round of Delphi approach. In the second round, the average opinion of the respondents about each of the criteria and components was estimated to be 88% and the standard deviation of their opinion was 20%; therefore, the final model with 12 criteria and 32 sub-criteria was confirmed and validated (Figure 1).

The results indicated that, according to the panel of experts, the quality of the nurse-patient relationship with a weight of 0.27 had the highest priority in evaluating the patient's experience with healthcare services, followed by the quality of the physician-patient relationship (0.241). Access to necessary medications ranked last (0.00) in terms of importance. Based on the results, the calculated consistency ratio for all decisions made by the expert panel was CR ≤0.1. Thus, all decisions exhibited adequate consistency (Figure 2).

Discussion

The purpose of the present study was to identifying criteria for evaluating patient experience of healthcare and prioritizing them using a combined Delphi and FAHP approach in the teaching hospitals of Mashhad University of Medical Sciences. Based on a literature review and criteria extracted from the Delphi approach, the criteria of how to communicate with nurses, providing nurses' services, providing doctor's services, how to communicate with doctors, participation of family and relatives, emotional support, physical environment and hoteling, how to discharge from the hospital, respect for Dignity and privacy of the patient, provision of information to the patient, access to necessary drugs and pain management were identified. Then, using the method of fuzzy hierarchical analysis process, the weight and importance of the components affecting the evaluation of the patient's experience was prioritized and determined.

Evaluation of Patient Experience Factors of healthcare

The findings of the present study highlight that nurse-patient communication with a weight of 0.279 holds paramount importance among various criteria of service quality. It is noteworthy that only the nursing group maintains a direct and extended interaction with patients within health-related contexts. The patient-caregiver relationship, central to nursing science, is grounded in professionalism, mutual respect, and trust (29). In another study, despite adequate facilities and equipment, patients expressed dissatisfaction with services, which was related to the disrespectful conduct of healthcare personnel (30). Effective communication serves as a linchpin for high-quality nursing care, resulting in heightened patient satisfaction and improved health.

The high quality of the interaction between nurses and patients leads to the high satisfaction of patients with nursing services. It increases the patient's readiness at discharge, and leads to decrease rate of returning to the hospital (31). In other words, it can improve the quality of nurse-patient interaction, causing a robust increase in patient satisfaction with the overall care providing by nurses. Since communication is a crucial aspect of nursing care, healthcare administrators must address these obstacles and devise solutions. Communication between nurse and patient will enhance the quality of services providing by nurses and is effective in diagnosing, treating, and enhancing the quality of nursing care.

According to the present study, physician communication (0.241) was the second most important factor influencing the patient's experience. However, in the study of Lathia and colleagues, the way to communicate with the physician was the first priority, and the way to communicate with the nurse was the second priority (32). The treatment of maladies highly depends on the interaction between physician and patient, which results in a quick and successful treatment (31). Numerous studies have demonstrated that the inability of medical staff, particularly physicians and nurses, to establish a proper relationship with patients not only results in high costs but also causes patients to be dissatisfied with the treatment process, refuse to follow treatment instructions, decide to replace their physician and have a negative attitude towards treatment and healthcare organizations. According to the findings of the present study, 34% of participants in a study investigating the perceived quality of hospital service ranked empathy as the most significant factor in the quality of healthcare services (33). Due to their fear, anxiety, and stress, patients want their physicians to attend to them and discuss the issues and complications of the disease and treatment. Similarly, it has been reported that a significant proportion of patient complaints about physicians and non-compliance with treatment orders are not due to the physician's negligence but rather is due to communication issues (34). According to another study, client satisfaction is the most important indicator of treatment quality. Based on an analysis of patient satisfaction with physician-patient communication, the maximum level of patient satisfaction with physician-patient communication was 63.4%, which may be attributable to physicians' poor communication skills or neglect of these skills (5). Considering the importance of interpersonal communication to the overall contentment of the patient, it is essential to include communication skills in the curriculum for medical students and the ongoing education of general and specialist physicians. The evaluation of students' communication skills is equally as essential as the evaluation of their scientific knowledge. Also, there should be sufficient monitoring about spending the necessary time for complete examination of patients, providing high-quality services, and providing complete explanations about the disease and treatment by physicians. Policymakers should pay attention to physicians' workload to have enough time to examine patients. Of course, this issue depends on matching the ratio of the physician to the patient admitted to the hospital (31). Also, in the evaluation and accreditation of medical centers, more emphasis should be placed on patients' satisfaction, especially through communication between medical staff and patients.

As shown in the present study, medical services as the third priority (0.175) effectively influence the patient's experience. The expertise, knowledge, and abilities of physicians and nurses are included in medical services. According to additional studies, medical services have always played a crucial role in promoting the integrity of hospital services. A patient visits the hospital primarily to receive quality services, such as a thorough examination and treatment from a physician (35). According to the findings of the present study, when assessing the quality of hospital services from the perspective of the patient's companions, the dimension of medical services (which included the indicators of accurate diagnosis of the disease, the results of treatments, and the knowledge and expertise of physicians) with a weight of 0.172was in the third category in priority setting of hospital services. The accurate diagnosis of the disease with a weight of 0.115 was the essential criterion for the hospital service quality evaluation. After that, the result of treatment with a score of 0.035, and finally, the knowledge and expertise of physicians with 0.022 were the following priorities (30). Patients have expectations that must be met, and the treatment outcome can ultimately contribute to trust in the physician. Since medical services are the primary process in a hospital system, managers and physicians are expected to perpetually evaluate these processes and work to enhance their quality to improve the quality of hospital services.

The influence of the physical environment and lodging on the patient experience with a weight of 0.012 rated ninth rank in priority. This low value was consistent with some other studies (35, 36). In a study that used the PROMETHEE technique, one of the multi-criteria decision-making methods for determining the best option in evaluating the quality of hospital services was communication with the physician and medical staff, physician knowledge and experience, and waiting time in the majority of hospital departments, which is consistent with our study (37). However, another study showed that hospital hoteling was a critical service quality criterion. Since the patient may be hospitalized for a long time, this criterion will significantly impact his/her experience (31).

The scores of the dignity and privacy of the patient (0.005), and access to the required drugs (0.000) were the lowest. The reasons, according to the experts, are: Patient privacy has been considered necessary in Iranian hospitals for many years. In this regard, a national project to provide same-sex clinical care was introduced and implemented in the country's hospitals, including: Patients hospitalized separately according to gender (men and women) in different departments. In most cases, medical services are provided by nurses of the same gender as patients. The patients can establish the necessary communication with the physician and the treatment staff in a private and comfortable space. Paraclinical services are also provided in separate spaces for male and female patients. In most cases, male and female employees provide services for male and female patients (except in cases where there are not enough employees or emergencies and crises). The studies of Alghamdi (38) and Ramez (39) also confirm these results in settings with religious conditions similar to Iran. However, it conflicted with a study that measured healthcare service quality from patients' perspectives: using the Fuzzy AHP application. The quality of services was evaluated by several criteria; the criterion of reliability was the most important. This criterion had several sub-criteria, and the creation of patient privacy was introduced as the second important sub-criteria (40). This difference in the results may be due to cultural and religious differences in the research community.

Hospitals and medical centers must provide all the medicines needed by inpatients; usually, a 24-hour pharmacy is open in hospitals to provide medicines for the patients. In a study that utilized a multi-criteria decision-making approach to examine the role of service quality measurement in the satisfaction of hospitalized patients, the effect of hospital pharmacy and access to necessary drugs was not assigned a weight (41). In the results of the second set of publications of the Health Care Quality Project of the Institute of Medicine in the United States regarding health care quality, this item were not explicitly examined (42). In the study of Bangoli et al.(30), access to medicine was assigned the lowest weight among the indicators related to access (0.024).

The present study had some limitations. Firstly, the complexity of the questionnaire occasionally necessitated guidance, which could have influenced the responses. Secondly, the evaluation was restricted to teaching hospitals of Mashhad University of Medical Sciences, raising concerns about the generalizability of the findings to other healthcare settings. Despite these constraints, the results of this study provide significant insights into improving the quality of health care. Therefore, it is recommended that future research endeavors focus on developing a questionnaire based on the proposed framework and incorporate the perspectives of patients regarding the importance of the identified criteria and gather their proposed solutions to enhance the patient experience within Iranian hospitals. By including the patients' viewpoints, future studies can provide a more comprehensive understanding of effective strategies for improving healthcare services.

Implications for practice

The substantial improvement of a patient's hospital experience of healthcare yields profound effects on service quality of the health system. Given the limited awareness of patients concerning the intricate criteria of hospital processes, the utilization of expert-driven decision-making methodologies, notably the FAHP approach, serves as an invaluable tool to discern the critical criteria that shape patients' experiences. Relevant managerial stakeholders can leverage these findings to effectively oversee and scrutinize patient experiences over specified intervals. This practice enables them to furnish constructive feedback to the workforce and senior management. Furthermore, it empowers them to craft and implement corrective measures designed to enhance the criteria of evaluation patient experience. In the realm of assessing the patient's service experience, the findings from the expert panel underscore the pivotal significance of effective nurse-patient and physician-patient communication, alongside the seamless provisioning of medical services. Consequently, the strategic employment of experienced and specialized nursing and medical personnel, accompanied by measures to facilitate comprehensive medical care, emerges as a requisite

Acknowledgments

The authors would like to thank the Vice Chancellor of Research Affairs, Student Research Committee and the heads and employees of the Mashhad University of Medical Sciences hospitals. The researchers are grateful to all participating experts in the hospitals covered by Mashhad University of Medical Sciences. We also thank Dr. Mohammad Miri (Non-communicable Disease Research Center, Department of Environmental Health Engineering, Sabzevar University of Medical Sciences for their cooperation in writing the article (technical editing, language editing and proofreading).

Conflicts of interest

The authors declared no conflict of interest regarding the publication of this study.

Funding

This research receives a grant from Mashhad University of Medical Sciences. The funding source had no involvement in design of the study, data collection, data analysis, etc.

Authors' Contributions

Zahra Keyvanlo: Conception and design, acquisition of data, interpretation of data, drafting and revising the article. Ali Vafaee-Najar, Elahe Hooshmand, and Fatemeh Kokabisaghi: Conception and design, analysis and interpretation of data, drafting and revising the article.  Monireh Ahmadimanesh: Conception and design, analysis and interpretation of data, drafting and revising the article. Mohammad Kargar: acquisition of data, drafting, and revising the article. Mojtaba Najib Jalaly: Conception and design, acquisition of data, drafting and revising the article. All authors contributed to the writing of the manuscript and discussed on the manuscript.