Evaluation of the Quality of Cancer-Related Fatigue Clinical Practice Guidelines

Introduction

Cancer-related fatigue (CRF) is a symptom that is commonly experienced by cancer patients regardless of the tumor type or its treatment (1). This state is the most commonly reported symptom in cancer patients (2). The National Comprehensive Cancer Network (NCCN) defines CRF as a subjective and sustained sensation that is physically and psychologically distressing and/or a cognitive fatigue or erosive activity, which interferes with the normal performance (3). The CRF affects both physical and mental abilities and is not relieved by rest (4). The management of CRF is a difficult measure in the clinical practices. The CRF can be one of the chemotherapy complications that exerts the most disturbing effects on the quality of life among patients (5).

Evidence-based care has an important role in improving the quality of nursing care due to facilitating the delivery of up to date care and being cost-effective for patients (6). Health system services are expected to be established based on evidence, as well as scientific methods and approaches (7). The implementation of evidence-based care guidelines is a solution to improve the quality of care and increase patient satisfaction (8). Therefore, clinical practice guidelines (CPGs) are considered to improve the quality of clinical practice by the reduction of inappropriate diversity, improvement of patient outcomes, minimization of loss or damage, and promotion of cost-effective measures (9, 10).

The CPGs are the documents including recommendations that are designed in accordance with the scientific indicators to facilitate decision-making in health care (5, 11). The achievement of effective health care requires the use of powerful tools (12) that can facilitate the adoption of evidence-based assessment and interventions for adult cancer patients who experience fatigue (13). The CPGs are among the important components of clinical performance, the quality of which affects the quality of care (14). Regarding this, the improvement of the quality ofCRF CPGs is an important issue in oncology.

High-quality CPGs facilitate more effective use of clinical services, reduce the undesirable variation of performance, and preclude the use of unnecessary services (15), thereby improving health outcomes (13). On the other hand, the abundance of the available CPGs in different disciplines is a concern because the recommendations generated by different groups can be varied or conflicting (16, 17). The high difference in the methodological quality of these CPGs poses concerns regarding the quality of their applied methods (18, 20-22). Consequently, some researchers do not recommend to use CPGs because they are unreliable, invalid, and unrelated (23).

However, it is well-recognized that the proper use of CPGs improves clinical performance (21). Given the fact that GCPs often have defects in terms of methodological quality (24), it is important to review the CPGs regarding their methodological strengths and limitations (25). Accordingly, it is required to adopt a strategy to differentiate between CPGs and ensure that they are implemented at the highest quality.

The Appraisal of Guidelines for Research and Evaluation (AGREE) collaboration network was established with the aim of improving the quality and effectiveness of the CPGs (18, 25). The AGREE instrument has three goals, namely evaluation of the methodological quality of CPGs, provision of a methodological approach to develop CPGs, and specification of what information and how information is presented in CPGs (26). This instrument can also be used to evaluate the quality of the CPGs that become a candidate for clinical use to formulate policy decisions or adapt recommendations from one context to another (25). The AGREE II instrument is widely used to evaluate the methodological quality of CPGs (26), including several cancer control-related CPGs.

The necessity of decreasing the burden of disease requires the availability of evidence-based CPGs (27). The assessment of CPGs is important to ensure the methodological rigor and the quality development of guidelines offering evidence-based recommendations for clinical practice (28). There is limited data on the availability, quality, and content of the nursing oncology guidelines in Iran. Additionally, there is a paucity of specific CPGs for nursing non-pharmacological interventions and CRF cases. Moreover, the CPGs should be adapted for nursing practices. With this background in mind, the present study aimed to determine the methodological quality of CPGs about CRF management using the AGREE II instrument.

Methods

This study involved a systematic search, followed by a descriptive study, to evaluate the methodological quality of CPGs about CRF in 2018. The evaluation of CPGs was accomplished via a systematic review method using the relevant studies (29-31). In most of the similar studies, no specific methods were adopted (10, 20, 22, 32-37). However, since the CPGs are not the primary resources, we used a systematic search, followed by a descriptive study.

The AGREE II instrument was used for the evaluation of CPGs. This tool consists of 23 items in 6 domains, including scope and purpose, stakeholder involvement, rigor of development, clarity of presentation, applicability, and editorial independence. The final section of the AGREE II instrument includes two other reviews that require the user to judge the quality of the CPG for overall assessment and state whether this CPG is recommended (31).

The scope and purpose domain (items 1-3) pertains to the guideline’s objectives, health questions, and target population (16). Stakeholder involvement criteria (items 4-6) indicates whether developed guidelines define a professional group or appropriate stakeholders, present the perspectives of intended users, and search patients' preference (28, 38). The rigor of development domain (items 7-14) relates to the methodology of gathering, briefing, and formulating evidence considered during guideline development, processes, and updating methods (16, 30).

Clarity of presentation domain (items 15-17) is related to structure, framework, and language of the guideline; in addition, it emphasizes the unambiguity and specificity of the recommendations (33). Applicability domain (items 18-21) concerns the facilitators and barriers to the application of the guideline recommendations, as well as the resource implications of implementing the guideline recommendations; in addition, it deals with economic factors affecting guideline implementation (25, 30). Finally, the editorial independence domain (items 22-23) pertains to ensuring the lack of bias in the guideline development and conflict of interest for all of the involved members and how it may have influenced the recommendation development process (30).

Each item of the AGREE II and the two global ranking items are graded on a 7-point Likert scale (ranging from absolutely disagree=1 [missing items] to totally agree=7 [expected quality]) (12, 21, 39). Five independent reviewers (MH, SS, SH, FT, and FHN) with 3-20 years of experience in cancer therapy and cancer research scored each guideline. We used the Farsi version of the AGREE instrument, the validity and reliability of which have been confirmed by Rashidain
et al. (40).

The inclusion criteria were: 1) English language, 2) adult target population (age of ≥18 years), 3) relevancy to cancer therapy-induced fatigue, 4) being introduced as a “guideline”, 5) involvement of recommendations about CRF, 6) being developed by institutions, communities, societies, associations, and cancer care groups, and 7) possession of guideline characteristics. On the other hand, documents similar to guidelines, such as systematic reviews, clinical pathways, protocols, instructional booklets, patients' guides, narrative reviews, and books, were excluded from the study.

For the purpose of the study, a systematic literature search was performed to find eligible published guidelines from the observation time until Jan 2018, using several databases and search engines, including Scottish Intercollegiate Guidelines Network, National Comprehensive Cancer Network, National Guideline Clearinghouse, Cumulative Index to Nursing and Allied Health Literature, National Institute for Clinical Excellence, Cancer Care Ontario, Guidelines International Network, Oncology Nursing Society, National Health Service Corps, Science Direct, Cochrane Library, Scopus, ProQuest, PubMed, Google Scholar, Google, and Yahoo.

In addition, the guidelines were search in a number of website, namely European Society for Medical Oncology, Canadian partnership against cancer, Agency Health Care Research and Quality, World Gastroenterology Organization, Cancer Care Manitoba, BMJ, and Best Practice. The search process was accomplished using several English keywords as follows: “Fatigue” AND “Cancer” OR “Chemotherapy” OR “Cancer therapy” AND “Recommendations” OR “Guideline” OR “Cancer-related Fatigue” and/or combinations of these keywords.

After finding the relevant documents, our research team screened their titles, abstracts, and full texts based on the eligibility criteria (Figure 1). Then, five reviewers independently appraised the methodological quality of the CPGs using the AGREE II instrument (the second version, updated in 2013) (41) based on its user manual (25). To this end, the appraisers evaluated each item and scored them from 1 (strongly disagree) to 7 (strongly agree) to measure the extent that each of the criteria was met by CPGs (42). Score 1 should be assigned when there is a lack of relevant information or this

information is reported very weakly, whereas score 7 is given if all of the mentioned criteria in the users' manual are addressed. In addition, the scores of 2-6 are assigned when the item of AGREE II instrument does not completely meet the criteria (43).

Each domain was scored by summing up the scores of its items. Then, the maximum and minimum possible scores were calculated for each domain. For instance, these values were measured for domain 1 (i.e., scope and purpose) as follows (Table 3) (21, 41):

Maximum possible score: 7 (strongly agree) × 3 (items) × 5 (appraisers)=105

Minimum possible score: 1 (strongly disagree) × 3 (items) × 5 (appraisers)=15

Furthermore, the standardized scores were calculated using the following equation (25):

(Obtained score–minimum possible score)/(maximum possible score–minimum possible score)

In addition, five reviewers appraised each included CPG, screened them independently, and gave an overall score for each of them. As mentioned earlier, this study was performed in an attempt to facilitate guideline adaptation process for our own nursing practices. In the current study, the median scores of < 30%, 30-60%, and > 60% were considered as indicating unfavorable, relatively favorable, and favorable quality levels, respectively, for each domain of CPGs (9).

In addition, in order to find the rigorously suitable CPGs about CRF, the quality of CPGs was evaluated according to the obtained percentage score of each domain. In this regard, if more than five domains of a guideline were scored 60%, it was strongly recommended; if more than four domains of a guideline were scored 30% and at least one domain was scored > 60%, it was weakly recommended; and if there were more than three domains scored < 30%, the guideline was not recommended (32).

Results

The search process resulted in the identification of 27 relevant documents, which were screened by the research team based on eligibility criteria. Since the present study was targeted toward the appraisal of the methodological quality of CPGs, the scientific content or recommendations of these CPGs were not shown. After screening the CPGs, two CPGs were included in the adaptation process (Figure 1). Table 1 presents the characteristics of CRF CPGs. Five appraisers evaluated the two CPGs using the AGREE II instrument. One of the two CPGs was evidence-based and the other one was an adapted CPG. Tables 2 and 3 indicate the mean scores of each domain of the two included guidelines assigned by the appraisers. The standard deviations of appraisers’ scores for each item of AGREE instrument were between 0 and 1, indicating the absence of diversity among appraisers’ evaluations.

The evaluated CPGs gained the highest score in the clarity of presentation domain and the lowest score in the applicability domain. The application of the implementation resources of the guideline was no discussed by any guidelines. Guideline 1 gained a higher score, compared to the other guideline. Based on the results, the clarity of presentation obtained the highest score (86.75%), followed by scope and purpose (85.5%), rigor of development (71.3%), stakeholder involvement (64.95%), editorial independence (35%), and applicability (15.45%). Therefore, the two evaluated CPGs obtained favorable scores in the clarity of presentation, scope and purpose, rigor of development, and stakeholder involvement domains. However, the other two domains (i.e., editorial independence and applicability) gained unfavorable scores.

The monitoring and auditing criteria were not presented in guideline 1; furthermore, it did not provide any advices or tools clarifying how the recommendations could be implemented (Table 2). The competing interests were not addressed in the two guidelines; in addition, the two guidelines didn’t describe facilitators and barriers to their implementations (tables 2, 3). Based on the findings, both of the investigated CPGs were weakly recommended. Although CPGs 1 and 2 obtained favorable scores in four domains, according to Li et al. (32), when only four domains scored above 60%, the CPG is weakly recommended.

Table 1. Summary and characteristics of cancer therapy-induced fatigue clinical practice guidelines

Table 2. Scores of each sub-domain of AGREE instrument by five appraisers for guideline 1

Source of AGREE instrument’s items: Brouwers M, Kho ME, Browman GP, Cluzeau F, Feder G, Fervers B, et al. on behalf of the AGREE Next Steps Consortium. AGREE II: Advancing guideline development, reporting and evaluation in healthcare. Can Med Assoc J. 2010b; 182: E839-E42. UPDATE: September 2013.

Table3. Scores of each subdomain of AGREE instrument by five appraisers for guideline 2

Source of AGREE instrument’s items: Brouwers M, Kho ME, Browman GP, Cluzeau F, Feder G, Fervers B, et al. on behalf of the AGREE Next Steps Consortium. AGREE II: Advancing guideline development, reporting and evaluation in healthcare. Can Med Assoc J. 2010b; 182: E839-E42. UPDATE: September 2013.

Discussion

The present study involved screening 27 relevant documents that lastly resulted in the evaluation of 2 guidelines. It happened since there is little CPGs about CRF, as we applied eligibility criteria. Other related studies also screened the similar related documents before to estimate the quality of CPGs (9, 29, 42).

In the current study, the methodological quality of CPGs about CRF was investigated by using AGREE II instrument. Based on the results, the scope and purpose domain gained a favorable score (median score: 85.5%), which is in line with the findings of other studies (23, 41, 44). This indicates that the CPGs sufficiently covered the guideline's objectives, health questions, and target population.

Consistent with other studies (9, 24), in the present study, the stakeholder involvement domain was estimated to be at a favorable level (median score: 64.95%). This signifies that the evaluated CPGs were efficient in considering a professional group or appropriate stakeholders, presenting the perspectives of the intended users, and searching patients' preference.

The present study showed the rigor of development domain (median score: 71.3%) was in a favorable condition. However, in a study conducted by Salarvand et al., this domain had the lowest score (9). In addition, in other studies, this domain was reported to have serious methodological defects in most of the appraised CPGs (45, 46). Cranney et al. showed that only a few osteoporosis CPGs were acceptable for application in their current format because the methodological quality of the evaluated CPGs was low (6). Given the importance of the rigor of development domain, indicating the methodology of gathering, briefing, and formulating evidence during guideline development, this domain should be addressed carefully.

In the present study, the clarity of presentation domain (median score: 86.75%) obtained the highest score. Similarly, in other studies, ‘scope and purpose’ and ‘clarity of presentation’ had the highest scores (over 80%) (29, 46). In addition, Chua et al. demonstrated that approximately all the guidelines obtained high scores in the domains of ‘scope and purpose’ and ‘clarity of presentation’ (9). This domain is related to structure, framework, and language of the CPGs and emphasizes on the unambiguity and specificity of the recommendations. The majority of the reported CPGs obtained relatively high scores in this domain (33, 40).

In the present study, the applicability domain had the lowest score. Accordingly, this domain is reported to be low (31) or  have the lowest score in other studies (12, 14, 22, 24, 28, 33, 46). Given the critical role of this domain in indicating the application of the guideline’s recommendations, strategies and resource implications for implementing the guideline’s recommendations, and presentation of economic factors affecting guideline implementation, it needs to be given more attention.

In the present study, the editorial independence domain obtained an unfavorable score (median score: 35%). Likewise, some studies reported a low score for this domain (12, 45). In contrast, this domain was reported to obtain a relatively high score in some other studies (29, 40). Editorial independence domain pertains to the issues investment, absence of bias in the guideline development, and conflict of interest; therefore, this domain should be addressed in the guideline development process.

 In the present study, the overall quality of the CPGs was reported to be relatively favorable by the appraisers (median score: 54.5%); therefore, it requires undertaking modification and improving the quality of CPGs. Similarly, Potting et al. stated that it is necessary to promote the methodological quality of CPGs for applying them in clinical practice (46).

Implications for Practice

As the findings of the present study indicated, the quality of the applicability and editorial independence domains were lower than the favorable level. Furthermore, the overall quality assessment score was rated as relatively favorable. Based on the findings, it is essential to improve the quality of CRF CPGs and design high-quality CPGs. In addition, the guideline developers should pay a particular attention to the two above mentioned subdomains, namely applicability and editorial independence.

Acknowledgments

The present study is a part of a PhD thesis, confirmed by the Research Deputy of Isfahan University of Medical Sciences, Isfahan, Iran (Code: 395651).We express our gratitude to the Deputy of Research for their funding support.

Conflicts of Interest

None-declared.