Designing and Evaluating Patient Education Pamphlets based on Readability Indexes and Comparison with Literacy Levels of Society

Introduction

To maintain and improve health status in health promotion programs, people are expected to play an active role in their own self-care. In order to assume the given role, individuals are required to receive adequate, relevant, and intelligible information to acquire the skills necessary for proper decision-making in terms of their health status (1, 2). Some parts of the information associated with the needs of people are provided through education. Moreover; in a healthcare system, patient education is among the priorities and standards of nursing care and it is also one of the essential components to promote and restore health and facilitate compatibility of patients with complications of diseases. Nowadays, patients are provided with maximum information by nurses through written education. Patient education materials i.e. pamphlets are of utmost importance in health promotion systems and nurses have focused their major educational activities on the contents of such pamphlets in order to direct the learning process of patients. Therefore, design and implementation of patient education pamphlets with an emphasis on individual conditions, disease-specific needs and situations are of paramount importance to generate the desired behavioral changes. Although written materials are the most effective and the least costly methods for patient awareness and education, they are often at a literacy level beyond that of patients (3) so most of patients do not have the ability to read at that level and consequently cannot properly understand healthcare information (1, 4).  This is one of the reasons for changes in patients’ behaviors in such a way that low levels of literacy among patients compared with readability level of educational pamphlets can be considered as a communication barrier in sending and receiving information to patients correctly and transfer completely different concepts in this respect.

The readability of a text written by the target group is an essential concept in the scope of patient education. Following design, corrections, and revisions; hundreds of patient education pamphlets are annually published in healthcare centers by experts and professionals and different indices are employed for their evaluation process. But it seems that majority of these indices are qualitative and general and do not benefit form enough transparency. Some of these manuscripts are written by students under the supervision of professors and experienced nurses and some others are developed by experts with relevant education, expertise, as well as theoretical and clinical skills. Therefore, the basic questions raised are what indices can be used in the evaluation of such materials? How such indices can become quantitative or judgments about educational contents for patients can be made objective? How the grounds of corrections and revisions can be based on objective and real research findings considering the direct and indirect expenses? In this respect, reviews showed that there are indices to accomplish such objectives which can be obtained through further investigations and validations. The application of readability indexes in order to change the level of pamphlets is based on the needs of society and the average level of literacy in society is likely to be affected in different periods. There are two techniques for the content analysis of educational materials including readability formulas and use of qualitative evaluation tools for written materials (5, 6).

Given the mentioned issues, there is a necessity to design patient education pamphlets based on teaching-learning standards and then to select and provide appropriate materials and contents according to the principles of content analysis. Researchers’ experience in the field of clinical education, importance of patient education in health promotion systems, and a sense of difficulty of materials in terms of their readability and understanding of messages to the patients were included as the motives to choose the subject of the present study. Thus, the main purpose of this study was evaluation and analysis of patient education materials i.e. pamphlets based on Fry’s Graph Readability Formula, Gunning Fog Index, Flesch Grade Level Readability Formula, Flesch-Kincaid Grade Level Formula, McLaughlin’s SMOG Readability Formula, Powers-Sumner-Kearl Readability Formula, Patient Education Materials Assessment Tool (PEMAT) checklist and their comparison with the level of readability in society in order to introduction several indicators for the use of nurses and experts as well as professionals in this field and provide useful feedback in terms of further promotion and greater effectiveness of patient education materials for designers and relevant experts.  

Methods

This study was conducted by using content analysis method based on readability indexes in Imam Ali (AS) Hospital and Imam Reza (AS) Hospital affiliated to Bojnurd University of Medical Sciences from February 2014 to February 2015. The term readability is a concept that merely shows the easiness or complexity of materials to read. This concept should not be mistaken with comprehension or understanding. Readability can be only determined through formulas which are related to the length of sentences,  number of words, and correct use of punctuations such as commas and points. To review the difficulty level of written materials, there are more than 40 formulas (7) out of which Fry’s Graph Readability Formula (8), Gunning Fog Index (9), Flesch Grade Level Readability Formula (10), Flesch-Kincaid Grade Level Formula (11, 12), McLaughlin’s SMOG Readability Formula (13, 14), Powers-Sumner-Kearl Readability Formula (15, 16), and PEMAT checklist (17, 18) designed based on standardized principles of patient education pamphlets were used following the results of previous studies evaluating patient education pamphlets (Table1).The validity of the PEMAT checklist was determined by using content validity method. To measure the reliability of the given checklist, 10 similar pamphlets were evaluated by two separate research assistants using the PEMAT checklist and its reliability was estimated by 0.84 via Cronbach’s alpha coefficient. The checklist is scored from zero to 20, which ultimately describes each pamphlet based on one of the given four grades: grade one is rated from 18 to 20 (it can be used unchanged), grade two is valued from 16 to 17 (it can be used after minor corrections and revisions), grade four is assessed from 13 to 15 (it can be used after major corrections and revisions), and grade four is scored below 13 (it cannot be not used). The formulas mentioned are considered as the main indexes of readability evaluation for patient education pamphlets (20, 22) which are also used to determine the level of readability in Persian (15, 16 and 23).

Table 1: Introduction of readability indexes

Figure 1: Fry’s Graph Readability Index

In this study; first, all the educational pamphlets used in both hospitals (n=69) were collected. Considering the fact that more than 90% of pamphlets employed in the hospitals in the city of Bojnurd were prepared without any Microsoft Office word file format or any other computerized format; words, syllables, and sentences were counted manually. Moreover, P for Text Software was used to evaluate a number of pamphlets in computerized format. Each pamphlet was evaluated independently by four raters and finally if there were discrepancies in counting of syllables, words, or sentences; they were re-evaluated to minimize the errors.

The sample size to evaluate patients’ literacy level was calculated equal to 500 patients according to the formula of  and based on a pilot study with d=0.25 and . According to this formula, the literacy level of 500 patients admitted to both hospitals were evaluated and their average was compared with the average level of readability for pamphlets.

In the end, based on the average level of literacy, all patient education pamphlets were corrected and revised. Then, their readability levels were examined based on McLaughlin’s SMOG Readability Formula and PEMAT checklist. At this stage, these two readability indexes were selected since the former had been employed as the most convenient method in most studies for teaching staff and the latter was consistent with the acceptable principles of the present research context. The SPSS Software (Version 20) was also used for data analysis.

Results

The most frequent subjects selected in patient education pamphlets included digestive system diseases (12 cases, 19.7%), and then diseases related to cardiovascular system (7 cases, 11.5%), respiratory system (6 cases, 9.8%), nutrition (6 cases, 9.8%), nervous system (5 cases, 8.2%), endocrinology (4 cases, 6.6%), urinary tract (4 cases, 6.6%), musculoskeletal system (4 cases , 6.6%), infections (4 cases, 6.6%), immunity (2, 3.3%), hematopoietic system (2 cases , 3.3%), ear-throat-nose (2 cases, 3.3%), breast (1 case, 1.6%), skin (1 case, 1.6%), mouth and tooth (1 case, 1.6%); respectively. The most frequent points included in the given pamphlets were associated with self-care (49 cases, 14%), definition of diseases (36 cases, 10.3%), disease symptoms (33 cases, 9.4%), causes of diseases (29 cases, 8.3%), treatments (26 cases, 7.4%), diagnoses (25 cases, 7.1%), prevention (24 cases, 6.8%), complications (24 cases, 6.8%), follow-ups (23 cases , 6.6%), diets (22 cases, 6.3%), drug regimens (18 cases, 5.1%), activities and exercises (14 cases, 4%), use of tools and equipment (12 cases, 3.4%), tests (11 cases, 1/3%) and the least frequent point was related to drug interactions (5 cases, 1.4%). It should be noted that each pamphlet contained one of the concepts mentioned and their frequency in the total number of pamphlets was calculated by 351 issues.

Table 2 shows the average level of literacy among patients admitted to the training hospitals in the present study. To this end, the literacy level of 500 patients admitted to both hospitals was recorded. The number obtained (6.72±4.34) was placed in grades six and seven i.e. elementary school according to readability indexes.

Table 2: Average level of literacy among patients admitted to Imam Ali (AS) Hospital and Imam Reza (AS) Hospital in the city of Bojnourd

In Table 3, the average level of readability for all pamphlets in the present study was determined based on the techniques of readability level. According to the guides, each number was interpreted and it was revealed that the majority of pamphlets were at college level. It should be noted that the number resulted from the formula specifies that the contents of the pamphlets or manuscripts are appropriate for which formal education classes in terms of level of readability. In this respect; Grades of Gunning Fog Index, Flesch-Kincaid Grade Level Formula, McLaughlin’s SMOG Readability Formula, and Powers-Sumner-Kearl Readability Formula were equivalent to formal classes in Iran. The level of Fry Readability Index is determined based on a graph which is the same as that by Gunning Fog. Categories in Flesch Grade Level Readability Formula are quite different and as the number in formula operation becomes greater, its level gets easier.

Table 3: Level of readability for pamphlets based on readability indexes

According to Table 4, 100% of pamphlets were at college level based on Flesch-Kincaid Grade Level Formula and Powers-Sumner-Kearl Readability Formula. In Flesch Grade Level Readability Formula, majority of pamphlets (79.7%) were at late college level and written at a very difficult level. According to McLaughlin’s SMOG Readability Formula, the bulk of pamphlets (91.3%) were placed at college level. Based on the PEMAT checklist, most of pamphlets (43.5%) scored below 13 i.e. grade 4.

As seen in Table 5, there was a considerable difference between the readability level of pamphlets before and after corrections and revisions according to existing standards based on McLaughlin’s SMOG Readability Formula and PEMAT indexes which was significant based on the results of paired t-test (p<0/05). McLaughlin’s SMOG Readability Formula showed that the majority (23.3%) of pamphlets were placed in grade seven after corrections and revisions. Furthermore, based on the results of the PEMAT checklist, the bulk of pamphlets (86.9%) were scored 18 to 20 i.e. grade one following the corrections and revisions to the pamphlets.

Table 4: Comparison of relative and absolute frequency of pamphlets based on readability levels in terms of individual readability indexes

Table 5: Comparison of the readability level of pamphlets before and after their corrections and revisions based on selected indexes

Discussion

In the present study, the readability level of preliminary pamphlets was written in very complex manner. Based on these findings, the average level of literacy among 500 patients in two training hospitals in the city Bojnurd was at grades six and seven while most pamphlets were at college level of literacy based on readability indexes. According to Gunning Fog Index, Flesch-Kincaid Grade Level Formula, and Powers-Sumner-Kearl Readability Formula; all the pamphlets were at college level. According to Powers-Sumner-Kearl Readability Formula, majority of pamphlets were written in late-college and very difficult levels. Given the results of McLaughlin’s SMOG Readability Formula, the bulk of pamphlets were at college level and majority of pamphlets scored lower than 13 i.e. grade four based on the PEMAT checklist. Such level of difficulty was also evident in studies conducted internationally. Spadero (1983) in a study evaluated 55 patient education pamphlets and found only 14 cases (25.5%) at readability level of 9 (24). Evaluating the level of readability for 190 patient education pamphlets in the United States and Canada, Lapierre and Mallet (1987) obtained an average level of 11.5±1.7 (25). Doak and Doak also obtained 7 for the literacy level of the population in their study while 10 was acquired on 100 selected sample of pamphlets which indicated a difference between the readability level of pamphlets in their statistical population and given standards (26).

All patient education programs are implemented since it is assumed that patients need to know and understand something in order to do learn and perform it.  Therefore, an effective patient education requires precise and calculated planning. The first stage in patient education planning is assessment of patients’ needs and abilities in a statistical population whose identification is an index for planning in a society. Under these conditions, it can be assured that patients seeking for education are motivated to change their behaviors and the educational programs have been effective. According to the related literature, patient education pamphlets associated with health education are often written in a very difficult level and they need one index in the study population to be changed. Although the basic indices have been identified in numerous research studies, any defined indices are based on the literacy level in the context in which the research has been conducted and it cannot be used as a fixed number in another population especially with a different language. Davis and others obtained 11 t0 14 for the readability level of patient education pamphlets in 1990, while the average level of readability within 5 months of admission to a hospital was 6 in that community. With a clear difference between the ability to read among people in the study and the level of educational materials, the researchers were required to make the readability level to the literacy level in society closer and finally the average level of readability for pamphlets reached 7 after corrections and revisions (27). In another study, Spadero (1983) estimated the average level of readability for pamphlets below 9 (24). Also, Lapierre and Mallet (1987) reported the average level of readability for pamphlets 8 or lower (25) and Farell, Mill, and Gontry (1989) reported the average level of readability below 8 (28). Matthews and Thornton (1985) also registered 6.5 to 8.5 for the average level of readability of pamphlets (29).

Considering that all the related studies in this field had been conducted abroad and due to the proposed variable numbers in different investigations, initially the average level of literacy among 500 hospitalized patients in a statistical population was obtained (6.72±4.34). Based on the interpretation of the above readability indexes, literacy level of the population studied was elementary schooling or a number between 6 and 7 which was consistent with the results of studies by Davis, Spadero, Lapierre and Mallet; Farell, Mill, Gontry; and Matthews and Thornton (24, 25, 27, 29). In this study, all the pamphlets were evaluated quantitatively based on six readability indexes as well as the PEMAT checklist. The average level of readability for pamphlets indicated that most of these pamphlets were written at college level and they were higher than the literacy level in the statistical population.

After correcting and revising the pamphlets, the average level of readability for pamphlets turned into 8.08±1.40 according to McLaughlin’s SMOG Readability Formula which revealed that the readability level had become closer to the literacy level in the population and significantly improved. Moreover, based on the results of the PEMAT checklist, the average readability level of pamphlets was 18.9±2.3 and it had reached to grade one from grade four after corrections and revisions showing that the pamphlets could be used unchanged. Such changes demonstrated improved quality of pamphlets in the given population in the present study.

Implications for Practice

Content analysis of patient education pamphlets not only contributes to their improved efficiency and effectiveness but also leads to accomplishment of desired self-care goals. For this purpose, recognizing the weaknesses and strengths of patient education pamphlets and their contents based on scientific and experimental (not mental) indices and improvement of the re-design of their structure can have a basic role in the systematic process of patient education and facilitate learning for patients. In order to fulfill the above indices and given the novelty of the study in Iran, it is expected that the results of this study help researchers of patient education in an effective manner.

Furthermore; considering that the study was conducted for the first time in Iran and researchers did not have any indexes to compare the readability level of pamphlets and the average level of literacy in society, the literacy was determined over a year on a population of 500 patients. Given the novelty of the present study, it is suggested that researchers conduct similar studies in different cities in order to obtain an accurate number for the average level of literacy in society at a national scope.

Acknowledgments

This study was a part of a research project approved with the code (no-93.41.1773) in the Faculty of Nursing and Midwifery in the city of Bojnurd (in northeastern Iran). We hereby appreciate all the managers and nursing staff in Imam Reza (AS) Hospital and Imam Ali (AS) Hospital affiliated to Bojnurd University of Medical Sciences for their cooperation in the fulfillment of this project.

Conflict of interest

The authors declare that there is no conflict of interest