External Validation of the Difficult Intravenous Access (DIVA3) Score as a Clinical Predictive Rule for Identifying Difficult Intravenous Access in Children

Introduction

Peripheral intravenous (PIV) catheter insertion is the most common painful procedure in pediatrics, and more than 90% of hospitalized children require it (1-3). Pediatric PIV access is a time-consuming clinical procedure. Children's veins are thin and fragile. On the other hand, dehydration, fever, and hemodynamic imbalances are common health problems in children that may affect access to peripheral veins and make it a challenging procedure for healthcare providers children (4). The success rate of the first attempt for PIV access decreases dramatically in some situations such as emergencies. Subsequent attempts impose more pain and stress on the child and parents (1, 5). According to evidence, frequent painful procedures in children result in psychological consequences, such as needle phobia and constant stimulation of the sympathetic autonomic neural system (6-8). For these reasons, PIV catheter insertion on the first attempt has become considerable in pediatric nursing literature. The success rate at the first attempt is reported as highly variable. It ranges between 53% and 75% (9). A failure in PIV placement at the first attempt is called "difficult PIV access" (10, 11). Determining factors which increase the failure access help clinicians to provide strategies such as applying available technologies which improve the access probability (9, 12). Some of the evidenced related factors of the failure include: prematurity, history of frequent hospitalization, child's age, skin shade, hemodynamic imbalance, and lack of cooperation (13-16). Developing a clinical predictive score is a common strategy to help healthcare providers prevent clinical consequences. Various techniques are available, such as using ultrasound or infrared radiation to increase the visibility of veins. These techniques can increase the likelihood of success (4, 6). 

Supporting the child and family in clinical settings and reduce their suffering are the professional responsibilities, a moral commitment, and an obligation of pediatric nurses in children’s healthcare (6, 17, 18). Healthcare providers have developed clinical rules, guidelines, and pathways to provide a-traumatic care for children. They aim for proper action with a low chance of success. In the case of PIV access, Yen et al. developed the Difficult Intravenous Access (DIVA) score (9, 19). DIVA is a clinical prediction rule for estimating the failure rate of PIV insertion in children. The Emergency Nurses Association (ENA) of the United States highly recommended this clinical rule due to its cost-effectiveness and feasibility to estimate the failure rate of PIV insertion in pediatrics (20). This clinical rule reduces unnecessary suffering of children and their parents, increases trust, and decreases the costs of repeated attempts for PIV cannula placement (6). At first, they proposed five variables including: vein palpability, vein visibility, age of a child, history of prematurity, and skin shade, then removed the last variable; and modified the scoring rule to four variables (19). Riker and colleagues (2011) modified the 4-variable version to their proposed 3-variable version and omitted the history of prematurity. The results showed that this 3-variable proposed version, like previous versions, could accurately predict the failure on the first attempt of vascular access. The score obtained is between 0 and 7, with a score of 4 or higher being 50% more likely to fail in the first attempt of IV cannula placement (21).

External validation of predictive tools with the aim of evaluating them is done by examining the reproducibility of results similar to the original development study in the target populations in other studies (22, 23). This study was performed aimed to test the external validity of the DIVA-3 score (cut-off point ≥ 4) in the children hospitalized in different wards of the two specialized pediatric hospitals in Iran.

Methods

Study design, setting and population

This prospective cross-sectional study was conducted on children hospitalized in the two educational specialized pediatric hospitals in Tehran from December 2020 to December 2021. In these hospitals, it is the responsibility of the nurses to place peripheral intravenous cannula. There is no protocol for clinical cases called "difficult IV."

Sample

A convenience sampling method was conducted. The inclusion criteria were: hospitalized children (28 days to 12 years) needed peripheral intravenous placement by nurses; not being in an emergency condition; catheterized with catheters between 20 and 26; their parents and nurses have willing to participate in the study. The children were admitted to receive different services according to the common policies of the children's hospital.

The exclusion criteria were: the refusal of the parent or nurse to continue the participation in the study; deciding to stop intravenous placement before the first attempt by the nurse.

In order to determine the sufficient sample size, "thumb rules" for external validation of clinical prediction models based on the entry of at least 100 events and 100 non-events were cited. Finally, due to the limitations related to the pandemic and the approval of the statistic consultant, the sampling statistics were completed with 230 non-events and 89 events.

Data collection

The researchers visited the studied hospitals during the morning shift at regular intervals. The clinical supervisors called the researcher if there was a need for PIV placement in any wards. Informed consent was obtained and then the checklist form of demographic information (child and nurse) was filled out. Thereafter, the nurse examined the child. After selecting the PIV place, the nurse scored the procedure of PIV access using DIVA3.

Measures

The tools used in this study were: (a) the form of demographic information of child and nurse and (b) the DIVA3 to estimate the successfulness of attempts for IV placement (table 1). The DIVA3 was scored as follows: Vein visibility after tourniquet (visible= 0, invisible=1), Vein palpability after tourniquet (palpable=0, impalpable = 1), age (less than 1-year =3, age of 12-35 months= 1, and ≥36 months = 0). Palpability and visibility are the two main criteria in determining the presence of a suitable vein for peripheral venous catheter placement by members of the health care team. These two items are scored on the DIVA scale with two modes: 0 meaning not visible or not palpable the vessel, and 2 meaning visible or palpable veins. The age of the child is scored with three points. Score two is given to the age group less than 12 months, which usually poses the biggest challenge to PIV insertion for health care providers. A score of zero is given to children over 36 months, in whom PIV insertion is easier. A score of 1 is given to children aged 12 and 36 months. The obtained score is between 0 and 7. A score of 4 or higher is 50% more likely to fail in the first attempt at venipuncture (20).  Since the phrases of this tool were short and simple, their translation into Persian was done by two researchers and approved by all team members.

Data analysis

The data were described using descriptive statistics. The measured outcome was failure or success of PIV cannula placement on the first attempt. The number of attempts was documented for all cases. The failure rates were determined for all children with DIVA SCORE of 4 and graters. The sensitivity, specificity, and positive and negative predictive values were determined considering a confidence interval (CI) of 95%. The success rates were obtained based on the hypothesized independent variables. Receiver Operator Characteristic (ROC) curve was used to show the diagnostic ability of the DIVA3 binary classifiers. The variables were compared using non-parametric tests, mainly the Chi-square test, in the two groups based on success/failure rate and their scores (categorized into less than four and equal or more than 4). The mean score was compared in the groups based on the study variables.

Ethical considerations

The study's aim and methods were explained to all nurses and parents of children who participated in the study. They were ensured of voluntary participation and confidentiality of the data, and their

Table 1. The difficult intravenous access (DIVA) score

verbal and written informed consents were secured.

Results

A total of 310 cases were analyzed in this study, of which 53.2% were male. The majority of children (56.5%) were in the age group of ≥ 3 years old. Neurologic health problems were the most reason for hospitalization in the cases (29.4%), and most of the children had a history of a chronic health problem disease for less than 12 months (58.4%).

The mean age of nurses performing IV placement was 34.17±6.56 years old. Their mean nursing experience was 9.42 ±5.98 years, and average experience as a pediatric nurse was 7.65±5.62 years. All nurses had a bachelor's degree in nursing, and the majority was female (88.3%). The size of the majority of IV cannula was 22 gauge (56.1%). Brachial (31.9%), radial (20.0%), and dorsal hand (16.6%) veins were the most common selected sites for the first attempt of IV placement.

The mean number of attempts was 1.87 (95% CI, 1.73-2.01), ranging from 1 to 9. The mean total score of DIVA3 was 2.48±2.36. The total score of DIVA3 was ≥4 for 89 cases (28.7%), and IV cannula placement failed on the first attempt for 75 cases (89.3%) of this group. Of the 221 children with a DIVA3 score <4, 14 cases (6.2%) had a failed IV cannula placement in the first attempt. The children less than 1-year-old had the most failure rate of IVP on the first attempt (71.3%).

The diagnostic accuracy measures for DIVA3 (cutoff point ≥4) were obtained. The cut off score usually is defined by the original author for the tool. The AUC for the DIVA3 in this study was 0.957 (95% CI 0.932 to 0.932). Sensitivity, specificity, Positive Predictor Value, and Negative Predictive value of DIVA3 (cutoff point ≥4) were 95.9%, 84.3%, 93.8, and 84.3%, respectively (95% CI 0.932 to 0.932) (Figure1).  In univariate analysis, the variables independently associated with failed IV cannula placement on the first attempt included: vein palpability, vein visibility, child's age, needle size, and nurse's experience (based on years working in the pediatric field). The variables' frequency and their association with failed IV cannula placement on the first attempt were listed in Table 2.

For 85 cases, two nurses scored at the same time using DIVA3 to calculate inter-rater agreement. Kappa values were between 0.52 and 0.89.

Figure 1. ROC curve of the DIVA3

Table 2. Mean scores of DIVA3 and first attempt success rate frequency based on the tested variables

*Chi-Square, **Kruskal-Wallis test

Discussion

In this study, the external validity of the DIVI3, as a clinical score, was investigated in the children admitted to the two specialized pediatric educational hospitals in Tehran. In these two settings, nurses are responsible for PIV placement.

The failure rate of PIV cannula placement on the first attempt was 28%. This result was similar to the value obtained by Girotto and colleagues (9). The failure rates had a wide range in different studies. In the studies of Shaukat et al. (13), Yen et al. (19), Riker et al. (21), and Keskin et al. (24), the failure rate was 41%, 14%, 32.2%, and 24.7%, respectively. The factors such as differences in children's demographic characteristics and clinical variables and the level of staff skills may influence the success rate of IV cannula placement.

The diagnostic accuracy measures for DIVA3 (cutoff point ≥ 4) support its discriminative power reported in the previous studies (21, 24, 25). According to Yen et al. (2008), a score ≥ 4 represents a 50% likelihood of IV placement failure on the first attempt. The obtained sensitivity and specificity of the DIVA4 (cutoff point ≥ 4) were 21.4 and 92.7, respectively (19). In another study, the calculation of AUC resulted in the amount of 0.72 for DIVA3. The sensitivity and specificity of DIVA3 were 56.8 and 22.2, respectively (21). The present study showed a higher diagnostic accuracy than the other studies. Sensitivity shows the probability that a tool/test score is positive when the disease exists (true positive rate) (25). Specificity shows the probability that the result of a test will be negative in the absence of disease or the target health status (true negative rate).  The DIVA's sensitivity represents the rate of failure in IV placement on the first attempt, while the score is four or above (positive for the target condition). Girotto and colleagues reported a sensitivity of 22.3% (95%CI. 14.7-31.6) and specificity of 93.5% (90.3- 95.9) for the DIVA3 (9). The sensitivity of DIVA3 in two other studies was reported as 75.9% (25) and 95.3% (13).

Despite debate about the adequacy of a nurse's clinical judgment in predicting difficult IVs, there is no doubt that a nurse's experience affects the clinical practice of healthcare providers. Studies have shown that clinical judgment alone is unreliable for decision-making (13, 25). In the present study, the failure rate of IV cannula placement on the first attempt was related to the work experience of the nurses. The similar result also has been reported in other studies (9, 19). Clinical prediction tools can help nurses estimate PIV access failure to take the interventions needed to increase the likelihood of success. These tools strengthen clinical judgment. Such clinical prediction tools like DIVA allow the opportunity to seek out the most skillful and experienced healthcare providers for IV cannula placement in difficult IV situations. Also, there will be provided criteria for choosing the application of alternative technology, such as infrared vein-finder lights or Venoscope (6, 19). The DIVA score still has limitations, such as wide variation in its diagnostic accuracy, so it is needed to develop higher discriminative tools and scoring rules. A more crucial issue is to improve overall success by adapting resources among children with DIVA scores ≥4 who need IV cannula placement. Highly skillful and experienced personnel for the pediatric settings, precise clinical pathways, and provision of technologic strategies such as ultrasound guidance are the resources evidenced in the literature (9, 13, 24).

In the present study, there was a significant relationship between the variables of child age, needle size, duration of disease, and nurses' experience with the failure rate of PIV placement, which supports the results of other studies (6, 9, 24). The results showed that the size of the PIV cannula and the duration of the child's disease impact the likelihood of failure. Consistent with the results of previous studies (9, 19, 21), success rate in PIV cannula placement on the first attempt were significantly correlated with nurses' experience. Increasing expertise means developing clinical judgment skills and thus a better chance of correctly estimating the probability of failure on the first attempt. On the other hand, increasing one's experience can improve clinical skills such as PIV cannula placement and the likelihood of success in the first attempt. According to some researchers (6, 21, 25), with increasing experience, especially work experience in pediatric wards, nurses' scores given to DIVA becomes more accurate; this limits the use of this tool for less experienced nurses. There are numerous factors influencing the success of PIV access, and their relationship is complex. So, success estimation is not possible based on only three variables. As long as the success of PIV access on the first attempt is of great value to pediatric nurses, more research is needed to develop predictive tools and use them in the clinic in the form of evidence-based guidelines or algorithms.

A recent study recommends that other factors such as age, education, level of expertise, work experience of the nurse, PIV location, type of disease, child's cooperation, and organization's management policies in this regard, all are valuable in predicting the success rate of PIV access. Therefore,  more accurate predictive tools are considerable (2). One of the limitations of the present study was the sampling method, which allowed only children who needed IV cannula placement during the day to enter the study. Second, the research setting was specialized pediatric hospitals, and the nurses who scored also had at least six months of experience working with children. Therefore, the results of this study cannot be generalized to children in need of venipuncture in non-specialized pediatric wards and by nurses who do not work in the field of children. One of the strength of the present study was that it is the first study in Iran on investigating the external validation of a clinical rule used in peripheral intravenous catheterization in children. The methodology can be considered by other nurse researchers.

Implications for practice

The results of this study support the importance of using DIVA3 despite the variable diagnostic value it has had in numerous studies. Using DIVA3 can help pediatric nurses to avoid unnecessary painful tries for IV catheter insertion.  Also, it would be helpful as a role in selecting between different veins. At the same time, it emphasizes the relationship between other factors and the need to consider them in the success rate prediction algorithm.

Acknowledgments

The researchers thank the nursing staff of the Mofid Children Hospital and Markaz Tebbi Kodakan for their willingness to participate in this study. This study was approved by Shahid Beheshti University of Medical Sciences, Tehran, Iran and was financially supported by the Shahid Beheshti University of Medical Sciences (1400/5/18/66004450). We would like to thank all participants who cooperated in the study, the authorities of the study setting which provided permission for conducting the study, and all people who helped us throughout the study. The Ethics Committee of Shahid Beheshti University of Medical Sciences, Tehran, Iran, approved the study (code: IR.SBMU.PHNM.REC.1399.144).

Conflicts of interest

The authors declared no conflict of interest.