Development and Psychometric Evaluation of the Comprehensive Nonverbal Pain Assessment Tool (CNPAT) in the Intensive Care Unit

Introduction

Pain is defined as an unpleasant sensory and psychological experience associated with damages. Pain involves various physical, physiological, and psychological dimensions that is a fundamental challenge in care systems (1). Pain in ICU patients with critical conditions and later stages of their life goes beyond physical pain and involves complete pain of physiological and psychological origin (2). Some of the causes of pain in ICU patients are related to nursing care, such as suctioning, changing position, drainage of secretions from connection tubes, deep breathing, and wound care. The prevalence of pain among patients in the ICU is reported to be 50% (3). About 87% of the patients in intensive care units experience moderate to severe pain (4). The failure to control pain in the ICU evokes disturbing memories of the length of the hospital stay (5) and causes anxiety and chronic pain for patients (6). Pain management in the ICU is a priority since the failure to treat pain increases mortality and results in many adverse physiological effects (5). Due to the importance of pain control and comfort promotion in ICU patients, pain management is considered a priority to prevent the adverse side effects of non-treatment, insufficient pain treatment, increase the quality of life, and reduce the unpleasant experience of the course of treatment. Accordingly, pain control is essential to patients, their families, and health care providers, so the healthcare organization has established some standards for pain management, including appropriate pain assessment tools for a timely and accurate pain assessment (7). The proper evaluation and management of pain in the ICU patients with cognitive impairment and inability to verbally communicate is challenging. According to critical and specific circumstances such as mental health disorders, the use of invasive devices and ventilators, sleep disorders, and patients’ mobility status, choosing the right tool to assess pain is of great importance (8-10).

The first step to effective pain management is choosing the right pain assessment tool (11). An effective pain assessment tool (PAT) for ICU patients is recommended to record pain scores in the form of a chart. Pain assessment record is essential for adjusting and adapting treatment efficacy to the patient’s needs; therefore, a standard recording format for  assessing pain of ICU patient is required to follow up treatment (12). Pain management by nurses with appropriate nonverbal PATs is essential to make the right decisions about the positive effects of palliative interventions and following up pain management (13). Due to the mental nature of pain, patients’ self-report is regarded as the gold standard for pain assessment; but self-reporting tools are not applicable in patients with an impaired level of consciousness. Therefore, using an appropriate tool to manage pain is essential in these patients (14). PATs have been developed in behavioral and behavioral-physiological symptoms dimensions in ICU patients with a reduced level of consciousness. There are several practical and standard PATs, including the Visual Analogue Scale (VAS), the Numeric Rating Scale (NRS) in patients who can self-report pain, the Behavioral Pain Scale (BPS) in ICU patients, and the Critical-Care Pain Observational Tool (CPOT) (15). The three Nonverbal Pain Scale (NVPS), CPOT, and BPS tools are pain scales commonly used in the ICU patients. The CPOT and BPS assess pain through behavioral assessment and measurement of the patient’s compatibility with a ventilator. At the same time, the NVPS performs a comprehensive pain assessment compared to other assessment tools by evaluating behavioral responses and examining physiological symptoms (4). In general, the behavioral-physiological symptoms dimension is not considered in the CPOT and BPS. Despite physiological symptoms, the internal validity (76%) and kappa coefficient (65%) are lower in the NVPS compared to the CPOT and BPS (3). Therefore, a comprehensive tool with excellent validity and reliability characteristics is needed to consider the behavioral and behavioral-physiological symptoms dimensions for patients unable to self-report pain.

The previous studies showed that 39.6% of nurses in the ICU do not know how to use pain assessment tools (PATs) and only 22% of them use pain assessment scales to estimate pain (4). Also, nurses’ performance after the intervention using the CPOT was significantly higher than before the intervention (12). Various studies have been conducted to evaluate nonverbal PATs in the ICU (12, 13). The CPOT and BPS have been considered as essential PATs. In the ICU chart sheets, pain is assessed using PAT and the quantitative facial expression is scored on a scale of 0 to 10. The PAT indicator measures pain intensity in chronic diseases such as low back pain (16). Due to its mental nature, there is no objective test for the assessment of pain, but in patients who are able to self-report pain, pain intensity can be measured with observational pain assessment tools such as PAT; however, it is inappropriate for patients unable to communicate verbally (17). Therefore, due to the inadequacy of PAT for pain assessment in ICUs and the importance of pain control, especially in patients with critical conditions who are unable to self-report pain, the present study was performed with aim to develop and psychometrically evaluate the Comprehensive Nonverbal Pain Assessment Tool (CNPAT) in ICU patients unable to self-report pain in order to assess pain and take timely action to control pain.

Methods

This methodological study was conducted following three phases from March to September 2023. The first phase of designing the Comprehensive Nonverbal Pain Assessment Tool (CNPAT) was done after review of the literature and available pain assessment tools. The second phase included evaluation of the psychological characteristics of the designed instrument (validity, reliability, sensitivity and specificity) and the third phase was the evaluation of comprehensive pain tool in patients with verbal inability. In the first phase, which was the review of articles published in Persian and English journals about pain assessment tools, 6 stages were performed. The first stage was designing research questions. What are the available tools for assessing patient pain in the ICU? The second stage was the selection of keywords related to the research topic and terms and planning to determine search strategies. It should be noted that descriptive terms and keywords were defined based on MeSH and according to the experts' opinion. Key words were tools, pain assessment, and intensive care unit. In the third stage, the inclusion and exclusion criteria were specified by the research team members. The inclusion criteria were: articles related to pain assessment tools, articles addressing the challenges of pain assessment in the intensive care unit, articles in English or Persian language, and articles which were reported in quasi-experimental, questionnaire and qualitative form. Exclusion criteria were the articles that only examined the advantages and disadvantages, the articles presented in the form of posters, speeches or letters to the editor and reviews which were not related to the research objectives. The fourth stage was a literature review of electronic databases, including Scientific Information Database (SID), Iran Medex, Iran Doc, and Magiran and international databases Science Direct, PubMed, Web of Science, Scopus, ProQuest, and Google Scholar by two researchers (S.k. and M.P.) independently based on keywords and predetermined strategies. Also, the sources of the reviewed articles were reviewed to access other articles. The fifth stage was the selection of qualified research articles. Abstracts of articles were reviewed by two researchers (first and second authors) and the screening of studies, extraction of results, and quality of articles were evaluated by two researchers independently. Related articles were separated and their full text was extracted. Overall 803 articles were found after removing duplicate articles. The articles entered the review stage in terms of title and abstract. After reviewing the titles and abstracts of the articles, in the second screening, in which the full text of the articles was reviewed and the articles were reviewed by two researchers based on the inclusion and exclusion criteria, 181 articles were remained.  The articles presented in the form of poster and speech at conferences and letter to the editor, case reports, and review article, and the articles that were not in the field of challenges in the implementation of the safe surgery checklist were excluded. Finally, 59 articles entered the final analysis. In the sixth stage, the quality of the articles was examined. The methodological quality of the articles was checked based on the tools used in various domestic and foreign studies. Finally 21 articles based on the purpose, the design of the study and inclusion and exclusion criteria were extracted (Figure 1).

According to a general review of studies, the most common pain assessment tools in the intensive care unit (ICU) were the three CPOT, BPS, and NVPS tools, and the two BPS and CPOT indicators were found to have superiority over the NVPS in both the intubated and non-intubated ICU admitted patients (3, 18). Then the translation and psychometric evaluation of the scale was done in six steps. In the first stage, the English version of the questionnaires was translated into Persian by two translators whose native language was Persian and had sufficient experience and mastery in translating English texts and tried not to change the meaning of the phrases. In the second step, the primary translations were combined into a single translation. At this stage, the original translated versions were reviewed and compared by experts and the contradictions between them were corrected and the original translations were merged. In the third stage, the translated version from the target language to the original language was reviewed in terms of the similarity of the concept, and finally the agreed English version was prepared. In the fourth stage, to obtain basic information about wording and existence of ambiguous points, the questionnaire was read and commented by 10 participants. In the fifth stage, the necessary amendments were made to the questionnaire and its final version was prepared for psychometric testing. In the sixth stage, all the translation and cultural matching procedures were documented. Then the items of the Comprehensive Nonverbal Pain Assessment Tool (CNPAT) was designed based on the NVPS, CPOT, BPS, Face-Leg-Activity-Cry-Consolability (FLACC), and Checklist of Non-verbal Pain Indicators (CNPI), Multi-dimensional Observational Pain Assessment Tool (MOPAT), and Nociceptive Coma Scale (NCS) instruments (1,19).

Figure 1: Flow diagram for literature review

The second phase of the study process included the evaluation of psychological characteristics of the designed tool (validity, reliability, sensitivity and specificity). Therefore, to develop a comprehensive and appropriate tool for patients unable to communicate verbally, the CPOT indicator was combined with the above mentioned assessment tools. Since the validity and reliability of the CPOT indicator have been confirmed in Iran, there was no need to do it again; therefore, to ensure its validity and reliability, the tool was developed and prepared after editing and deleting duplicate items. The developed comprehensive tool was scored on a scale of 0 to 15. Also, CPOT, NCS, FLACC, MPAT, CNPI, BPS included 33 items that 19 items were removed due to duplication. The 14 items include: CPOT (4 items) + 10 items reminded. In the qualitative content validity stage, the opinions of a 10-member panel of experts including ten nursing professors of the School of Nursing and Midwifery having expertise in the field of critical care and medical-surgical nursing in the hospitals affiliated to the Shahid Beheshti University of Medical Sciences were used. The items of the tool were examined in terms of grammar, meaning, and convenient location. Based on the views of the panel group and the removal of duplicate items, the developed tool containing seven items was obtained. Then 7 items were removed due to lack of approval by panel of experts as an important factor in pain assessment. Finally, 7 items include: CPOT + 3 items introduced as a comprehensive tool for pain assessment. Total score of the tool was 25. Then the tool with 7 items was evaluated by the content validity index (CVI) and content validity ratio (CVR). The same 10-member panel of experts was asked to express their opinions on the necessity of the tool items (CVR) and determining the relationship among the items and the intended concept (CVI). The content validity index for the whole scale was 0.9 and for the scale items was 0.95. In related CVR of the scale items scoring based on item relevance was 0.9 and no items were deleted. To assess the tool's reliability, the two Cronbach’s alpha and test-retest (ICC) methods were used (Cronbach’s alpha = 0.8, ICC = 0.76). Also, to determine the sensitivity and specificity of the device, the receiver operating characteristics (ROC) curve was used. Friedman test was used to compare the items of the developed tool in three modes: before the intervention (suctioning), during the intervention, and 20 minutes after the intervention. If the cut-off point was 4, the specificity of the tool was estimated at 94%, and if the cut-off point was 5, the specificity of the tool was estimated at 97.5%. Moreover, the sensitivity of the tool was estimated at 99%.

The third phase of the study was evaluation of comprehensive pain tool in patients with verbal inability. In this phase, 80 patients admitted to ICUs in the period of six months in 5 hospitals affiliated to the Shahid Beheshti University of Medical Sciences were included. The subjects were sampled by convince sampling, taking into account inclusion and exclusion criteria. Regarding patients’ conditions and their inability to make decisions, written informed consent was obtained from one of their first-degree family members to participate in the study. Inclusion criteria were patients with GCS  <8 aged 18 years and older admitted to ICUs who could not self-report pain  and failure to use the sedation during the intervention by the Richmond score. Exclusion criteria were brain damages leading to the paralysis of four limbs or both legs in hospitalization and the use of muscle relaxants such as Atracurium, Succinylcholine, Pancuronium, Cisatracurium, and mivacurium on medical prescription. Data were analyzed using SPSS software (version 19). Friedman test was used to compare the data in three modes: before, during the intervention (suctioning), and 20 minutes after the intervention.

Ethical Consideration

One of the important points of this new study was the use of a comprehensive pain assessment protocol for nurses. Also, the use of non-random sampling method and the non-standard conditions of sensory and sound stimulation in this research were points that can affect the results. The study protocol was approved by the Ethics Committee of the Clinical Development Unit of Loghman Hakim Hospital, Tehran, Iran (ethics code: IR.SBMU.RETECH.REC.1400.1226, research proposal code: 32293). 

Results

The results of the present study showed that 50% of the participants were over 60 years of age (age range 20-92 years); also, the mean age of the participants was 64.45±17.81 years. The mean length of hospital stays of the patients was 12±1.4 days. According to Table 1, 61% of the patients were male and 39% were female. In addition, 45% of the patients were hospitalized due to lung diseases, 7% due to sepsis, and the other cases were related to surgery, trauma, and other conditions. The Richmond Agitation Sedation Scale (RASS) was reported to be above zero in 53.8%, equal to zero in 35%, and ranged from 0 to 3 in 11.3% of the patients (Table 1).

Using the Friedman test, each item was implemented in three states before, during and after the intervention on a group of patients, and then the mean in these three time periods were checked and the test result indicated an increase in the scores. The designed index was a decrease during the intervention and then 20 minutes after the intervention.

The total mean score in the three intended modes was 3.8±1.62, 9.83±2.61, and 3.5±1.45 before the intervention, during suctioning, and 20 minutes after the intervention, respectively (p<0.001) (Table 2).

Estimated Marginal Mean (EMMs) plots showed that the difference in the total score in the three intended modes indicated an increase in pain score during suctioning and then a decrease in pain score after applying a painful stimulus (Figure 2). A decrease, an increase, and then a decrease in the indicator score were well represented by the EMMs plot. These states were associated with three modes including during the assessment, during the intervention, and after the intervention, respectively.

Table 1: Demographic and medical characteristics of the participants

Table2: The results of Friedman test before, during and after the intervention (suctioning)

Figure 2: Evaluation of the items using the EMMs chart

Table 3: CNPAT (Comprehensive Nonverbal Pain Assessment Tool) Score in ICU patients

Discussion

The purpose of the present study was to develop and psychometrically evaluate the Comprehensive Nonverbal Pain Assessment Tool (CNPAT) in the patients admitted to the intensive care unit (ICU) who could not self-report pain for comprehensive assessment and timely diagnosis of pain. In this research scales such as CPOT, NCS, FLACC, MPAT, CNPI, and BPS were reviewed and each of the scales had specific strengths and weaknesses. For example, CPOT and FLACC are the behavioral pain assessment scale used for nonverbal or preverbal patients who are unable to self-report their level of pain, but these two scales have no item regarding the patient's vital signs, which can be a warning for the patient's pain with a decrease in the level of consciousness. NCS scale is an observational pain tool that is used for patient with disorders of consciousness due to acquired brain injury (ABI). The mPAT is an observational scale designed to assess neonatal pain. BPS scale included three main parts of face status, movement of upper limb, and moaning in the non-intubated patients/patients under mechanical ventilation. Although each of these tools has advantages and disadvantages, the proposed comprehensive tool is extracted from the integration of these tools and covers all the items in these mentioned tools and can correctly measure the pain level of patients with reduced level of consciousness. In the present study, application of this tool in patients with a decreased level of consciousness before, during and after suction showed that the increase in mean pain during suction indicated that the tool was able to check the pain of patients during the intervention compared to before and after suction.

In 2018, a study entitled “The comparison of an observational pain assessment tool and physiological indicators in ICU patients under mechanical ventilation” was conducted in India. This retrospective observational study was performed on the ICU patients diagnosed with sepsis. The two processes of chip suction and position change were used to assess pain by comparing the two CPOT and physiological symptoms criteria. So that pain assessment and comparison of CPOT chart and physiological symptoms were performed in resting position, during the intervention, and after the intervention. Pearson’s correlation coefficient showed a trivial insignificant relationship between the two criteria. Also, their results indicated that the CPOT tool is suitable for assessing pain in ICU patients who cannot communicate verbally (20). However, in the present study, according to the opinion of experts and staff experienced in the ICU regarding the fact that physiological symptoms are still used as a symptom of pain, we decided to consider physiological symptoms in pain assessment.

In fact, the NVPS has two observational and physiological dimensions. Its difference with the CPOT lies in measuring physiological parameters of blood pressure, the number of heartbeats and breaths, and the behavioral observational pain symptoms. The findings of the present study indicated that according to the studies reviewed, physiological symptoms help assess pain in patients unable to communicate; however, caution should be taken when a nurse uses the tool to assess pain because the factors such as anxiety, restlessness, and infection alter physiological parameters in patients (5). In the present study, concerning the opinions of experts and nursing staff regarding the fact that physiological symptoms along with the factors involved in pain assessment facilitate timely diagnosis of pain in patients unable to self-report pain, the items of CNPAT as a combined valid and reliable tool were designed according to physiological symptoms after making writing corrections and ensuring the validity of the tool.

A study entitled “Pain measurement in patients under mechanical ventilation: behavioral pain scale vs. observational pain instrument” was conducted on ICU patients. The BPS score increased by one point and the CPOT score did not change as the resting position changed and the painful process was performed, while both the BPS and CPOT scores increased by two points by changing the resting place and completing the task non-painful process. The internal validity of the two CPOT and BPS tools with 95% confidence limits interval was regarded as good. Their results also showed that the CPOT was preferable to the BPS, and the BPS had a lower performance than the CPOT due to the change in its score caused by non-painful stimuli (1). Therefore, in the present study, the tool was designed based on the CPOT items. In combination with the intended indicators, similar items with CPOT items were removed in the study's first phase. The validity of the developed tool can be regarded as representative of an excellent validity and appropriateness of an instrument.

A study entitled “The psychometric evaluation of three behavioral pain assessment tools in patients with special conditions unable to self-report pain” was conducted in Chicago. The inclusion criteria were age over18 years, Richmond Agitation Sedation Scale (RASS) score above -4, inability to self-report pain, and lack of deep sedation. The evaluation of the three tools (CPOT, BPS, and NVPS) was performed in three modes: during resting and before the implementation of painful processes such as position change and suctioning, during the performance of the processes, and 10 minutes after implementing the procedures. The internal validity of the three instruments (CPOT, BPS, and NVPS) was measured using the Kappa reliability coefficient. The Kappa coefficient indicated excellent internal validity for the CPOT and BPS and good internal validity for the NVPS. The internal reliability assessed using Cronbach’s alpha indicated higher reliability of CPOT and BPS than NVPS. Their results showed an appropriate psychometric evaluation of the three instruments. Moreover, BPS and CPOT were found to be more beneficial for both the intubated and non-intubated ICU patients as compared to NVPS due to their poor internal reliability in non-intubated patients (3). In the present study, the two Cronbach’s alpha and test-retest methods were used to determine the instrument's reliability, indicating good reliability.

In this regard, Warden et al. conducted a study entitled “Development and psychometric evaluation of the pain assessment in advanced dementia”. The tool developed in their study has been regarded as a suitable simple representative of the pain assessment tool in the study population as its validity and reliability were ensured (21). The purpose and methods of the present study are similar to their study. However, the only difference is that the present study was performed on ICU patients and the tool was developed based on the CPOT indicator in combination with valid and reliable indicators designed for ICU patients. In general, it can be said that the comprehensive pain measurement tool designed in this research examines 10 dimensions, including restlessness, vital signs, facial appearance, and other issues, and can correctly measure the patient's pain with decreased consciousness. It is suggested that it should be examined in a comprehensive study with a larger sample size, and if approved, the present tools should be added and used in the ICU sheets.

Implications for practice

The results of the present study suggest that the Comprehensive Nonverbal Pain Assessment Tool (CNPAT) is appropriate for pain assessment in patients who cannot self-report pain.

Acknowledgments

The authors are grateful from the Clinical Research Development Unit of Loghman Hakim Hospital and Shahid Beheshti University of Medical Sciences for their support and cooperation.

Conflicts of interest

The authors declared no conflict of interest.

Funding

This research receives a grant from Shahid Beheshti University of Medical Sciences.

Authors' Contributions

Marzieh Pazokian: conceptualization, methodology, supervision, and writing the original draft. Farzaneh Khalandi: conceptualization, data collection, and writing the original draft. Sorour Khari: data collection, and writing the original draft. Neda Saniae: formal analysis, and writing the original draft. All authors contributed to the writing of the manuscript and discussed on the manuscript.