Designing and Evaluating Patient Education Pamphlets based on Readability Indexes and Comparison with Literacy Levels of Society

Document Type: Original Quantitative and Qualitative Research Paper

Authors

1 MSc, Faculty of Nursing and Midwifery, North Khorasan University of Medical Sciences, Bojnurd, Iran

2 Assistant Professor, Evidence Based Care Research Centre, Department of Pediatric Nursing, School of Nursing and Midwifery, Mashhad University of Medical Sciences, Mashhad, Iran

3 Tabatabaeichehr

Abstract

Background: Hundreds of patient education materials i.e. pamphlets are annually published in healthcare systems following their design, correction, and revision.
Aim: to design and evaluate patient education pamphlets based on readability indexes and their comparison with literacy level in society.
Method: The average literacy level among 500 patients admitted to two training hospitals in Bojnurd (northeastern Iran) was determined in 2014-2015. Afterwards, all patient education pamphlets in both hospitals (n=69) were collected and their readability level was determined. After that, all the pamphlets were re-designed according to the given standards and in line with literacy level in society. The SPSS software (Version 20) was also used to analyze the data.
Results: The average level of literacy among 500 patients in both hospitals in the present study was 6.72±4.34 which was placed in grades six and seven in terms of the guide to readability indexes. In line with McLaughlin’s SMOG Readability Formula, the bulk of pamphlets (91.3%) were at college level before corrections and revisions based on the given standards, but 23.2% were at a level lower than grade seven following corrections and revisions.
Implications for Practice: Evaluation of patient education pamphlets plays an important role in promoting self-care among patients. Due to the novelty of the present study in Iran, the results of this study can contribute to patient education researchers in order to identify the strengths and weaknesses of patient education materials i.e. pamphlets based on scientific indices as well as their revisions and re-developments.

Keywords

Main Subjects


Introduction

To maintain and improve health status in health promotion programs, people are expected to play an active role in their own self-care. In order to assume the given role, individuals are required to receive adequate, relevant, and intelligible information to acquire the skills necessary for proper decision-making in terms of their health status (1, 2). Some parts of the information associated with the needs of people are provided through education. Moreover; in a healthcare system, patient education is among the priorities and standards of nursing care and it is also one of the essential components to promote and restore health and facilitate compatibility of patients with complications of diseases. Nowadays, patients are provided with maximum information by nurses through written education. Patient education materials i.e. pamphlets are of utmost importance in health promotion systems and nurses have focused their major educational activities on the contents of such pamphlets in order to direct the learning process of patients. Therefore, design and implementation of patient education pamphlets with an emphasis on individual conditions, disease-specific needs and situations are of paramount importance to generate the desired behavioral changes. Although written materials are the most effective and the least costly methods for patient awareness and education, they are often at a literacy level beyond that of patients (3) so most of patients do not have the ability to read at that level and consequently cannot properly understand healthcare information (1, 4).  This is one of the reasons for changes in patients’ behaviors in such a way that low levels of literacy among patients compared with readability level of educational pamphlets can be considered as a communication barrier in sending and receiving information to patients correctly and transfer completely different concepts in this respect.

The readability of a text written by the target group is an essential concept in the scope of patient education. Following design, corrections, and revisions; hundreds of patient education pamphlets are annually published in healthcare centers by experts and professionals and different indices are employed for their evaluation process. But it seems that majority of these indices are qualitative and general and do not benefit form enough transparency. Some of these manuscripts are written by students under the supervision of professors and experienced nurses and some others are developed by experts with relevant education, expertise, as well as theoretical and clinical skills. Therefore, the basic questions raised are what indices can be used in the evaluation of such materials? How such indices can become quantitative or judgments about educational contents for patients can be made objective? How the grounds of corrections and revisions can be based on objective and real research findings considering the direct and indirect expenses? In this respect, reviews showed that there are indices to accomplish such objectives which can be obtained through further investigations and validations. The application of readability indexes in order to change the level of pamphlets is based on the needs of society and the average level of literacy in society is likely to be affected in different periods. There are two techniques for the content analysis of educational materials including readability formulas and use of qualitative evaluation tools for written materials (5, 6).

Given the mentioned issues, there is a necessity to design patient education pamphlets based on teaching-learning standards and then to select and provide appropriate materials and contents according to the principles of content analysis. Researchers’ experience in the field of clinical education, importance of patient education in health promotion systems, and a sense of difficulty of materials in terms of their readability and understanding of messages to the patients were included as the motives to choose the subject of the present study. Thus, the main purpose of this study was evaluation and analysis of patient education materials i.e. pamphlets based on Fry’s Graph Readability Formula, Gunning Fog Index, Flesch Grade Level Readability Formula, Flesch-Kincaid Grade Level Formula, McLaughlin’s SMOG Readability Formula, Powers-Sumner-Kearl Readability Formula, Patient Education Materials Assessment Tool (PEMAT) checklist and their comparison with the level of readability in society in order to introduction several indicators for the use of nurses and experts as well as professionals in this field and provide useful feedback in terms of further promotion and greater effectiveness of patient education materials for designers and relevant experts.  

 

Methods

This study was conducted by using content analysis method based on readability indexes in Imam Ali (AS) Hospital and Imam Reza (AS) Hospital affiliated to Bojnurd University of Medical Sciences from February 2014 to February 2015. The term readability is a concept that merely shows the easiness or complexity of materials to read. This concept should not be mistaken with comprehension or understanding. Readability can be only determined through formulas which are related to the length of sentences,  number of words, and correct use of punctuations such as commas and points. To review the difficulty level of written materials, there are more than 40 formulas (7) out of which Fry’s Graph Readability Formula (8), Gunning Fog Index (9), Flesch Grade Level Readability Formula (10), Flesch-Kincaid Grade Level Formula (11, 12), McLaughlin’s SMOG Readability Formula (13, 14), Powers-Sumner-Kearl Readability Formula (15, 16), and PEMAT checklist (17, 18) designed based on standardized principles of patient education pamphlets were used following the results of previous studies evaluating patient education pamphlets (Table1).The validity of the PEMAT checklist was determined by using content validity method. To measure the reliability of the given checklist, 10 similar pamphlets were evaluated by two separate research assistants using the PEMAT checklist and its reliability was estimated by 0.84 via Cronbach’s alpha coefficient. The checklist is scored from zero to 20, which ultimately describes each pamphlet based on one of the given four grades: grade one is rated from 18 to 20 (it can be used unchanged), grade two is valued from 16 to 17 (it can be used after minor corrections and revisions), grade four is assessed from 13 to 15 (it can be used after major corrections and revisions), and grade four is scored below 13 (it cannot be not used). The formulas mentioned are considered as the main indexes of readability evaluation for patient education pamphlets (20, 22) which are also used to determine the level of readability in Persian (15, 16 and 23).

Table 1: Introduction of readability indexes

Fry’s Graph Readability Index (8)

  1. Select three 100-word passages from the initial, middle, and final sections of the evaluated manuscript at random
  2. Count the number of sentences available at each 100-word passage
  3. Count the number of existing syllables at each 100-word passage
  4. Calculate the average number of sentences available for three 100-word passages
  5. Calculate the average number of syllables available for three 100-word passages
  6. Determine the readability level based on the chart of Fry’s Graph Readability Index (Figure 1).

Gunning Fog Index (9)

  1. Select three 100-word passages from the initial, middle, and final sections of the evaluated manuscript at random
  2. Count the number of sentences available at each 100-word passage
  3. Determine the average sentence length (ASL) through dividing the number of words by the number of complete sentences for each 100-word sample
  4. Count the number of three-syllable words and so for hard words available and  at each 100-word passage
  5. Sum up the number of hard words with ASL
  6. Determine the readability level of the manuscript based on Gunning Fog Index formula= (ASL+Hard Words)×0.4
  7. Place the number obtained in the table and determine the level of easiness and complexity for the manuscript

Flesch Grade Level Readability Formula (10)

  1. Select three 100-word passages from the initial, middle, and final sections of the evaluated manuscript at random
  2. Count the number of sentences available at each 100-word passage
  3. Count the number of existing syllables at each 100-word passage
  4. Determine the ASL of the three 100-word passages
  5. Calculate the average of syllables per word (ASW) through dividing the number of syllables by the number of words for three 100-word passages
  6. Determine the readability level of the manuscript based on Flesch Grade Level Readability formula= 206.835-1.015(ASL)-84.6(ASW)
  7. Place the number obtained in the table and determine the level of easiness and difficulty for the manuscript

Flesch-Kincaid Grade Level Formula (11, 12)

  1. Select three 100-word passages from the initial, middle, and final sections of the evaluated manuscript at random
  2. Count the number of sentences available at each 100-word passage
  3. Count the number of existing syllables at each 100-word passage
  4. Determine the average sentence length (ASL) for three 100-word passages
  5. Calculate the average of syllables per word (ASW) for three 100-word passages
  6. Determine the readability level of the manuscript based on Flesch-Kincaid Grade Level Formula = 0.39(ASL)+11.8(ASW)-15.59
  7. Place the number obtained in the table and determine the level of easiness and complexity for the manuscript

McLaughlin’s SMOG (Simple Measure of Gobbledygook) Readability Formula (13,14)

  1. Select three 10-sentence passages from the initial, middle, and final sections of the evaluated manuscript at random
  2. Count the number of three-syllable words and so for hard words available and  at each 10-word passage
  3. Determine the readability level of the manuscript based on McLaughlin’s SMOG Readability Formula

Powers-Sumner-Kearl Readability Formula (15, 16)

  1. Select three 100-word passages from the initial, middle, and final sections of the evaluated manuscript at random
  2. Count the number of sentences available at each 100-word passage
  3. Count the number of existing syllables at each 100-word passage and calculate the mean
  4. Calculate the average of syllables per word (ASW) for three 100-word passages
  5. Determine the readability level of the manuscript based on Powers-Sumner-Kearl Readability Formula=(ASL×0.0788)+(average of syllables×0.0455)-20.2026

Patient Education Materials Assessment Tools (PEMAT) checklist (17, 19)

  1. PEMAT checklist includes 20 items which is scored totally from 0 to 20

 

 

 

Figure 1: Fry’s Graph Readability Index

 

In this study; first, all the educational pamphlets used in both hospitals (n=69) were collected. Considering the fact that more than 90% of pamphlets employed in the hospitals in the city of Bojnurd were prepared without any Microsoft Office word file format or any other computerized format; words, syllables, and sentences were counted manually. Moreover, P for Text Software was used to evaluate a number of pamphlets in computerized format. Each pamphlet was evaluated independently by four raters and finally if there were discrepancies in counting of syllables, words, or sentences; they were re-evaluated to minimize the errors.

The sample size to evaluate patients’ literacy level was calculated equal to 500 patients according to the formula of  and based on a pilot study with d=0.25 and . According to this formula, the literacy level of 500 patients admitted to both hospitals were evaluated and their average was compared with the average level of readability for pamphlets.

In the end, based on the average level of literacy, all patient education pamphlets were corrected and revised. Then, their readability levels were examined based on McLaughlin’s SMOG Readability Formula and PEMAT checklist. At this stage, these two readability indexes were selected since the former had been employed as the most convenient method in most studies for teaching staff and the latter was consistent with the acceptable principles of the present research context. The SPSS Software (Version 20) was also used for data analysis.

 

Results

The most frequent subjects selected in patient education pamphlets included digestive system diseases (12 cases, 19.7%), and then diseases related to cardiovascular system (7 cases, 11.5%), respiratory system (6 cases, 9.8%), nutrition (6 cases, 9.8%), nervous system (5 cases, 8.2%), endocrinology (4 cases, 6.6%), urinary tract (4 cases, 6.6%), musculoskeletal system (4 cases , 6.6%), infections (4 cases, 6.6%), immunity (2, 3.3%), hematopoietic system (2 cases , 3.3%), ear-throat-nose (2 cases, 3.3%), breast (1 case, 1.6%), skin (1 case, 1.6%), mouth and tooth (1 case, 1.6%); respectively. The most frequent points included in the given pamphlets were associated with self-care (49 cases, 14%), definition of diseases (36 cases, 10.3%), disease symptoms (33 cases, 9.4%), causes of diseases (29 cases, 8.3%), treatments (26 cases, 7.4%), diagnoses (25 cases, 7.1%), prevention (24 cases, 6.8%), complications (24 cases, 6.8%), follow-ups (23 cases , 6.6%), diets (22 cases, 6.3%), drug regimens (18 cases, 5.1%), activities and exercises (14 cases, 4%), use of tools and equipment (12 cases, 3.4%), tests (11 cases, 1/3%) and the least frequent point was related to drug interactions (5 cases, 1.4%). It should be noted that each pamphlet contained one of the concepts mentioned and their frequency in the total number of pamphlets was calculated by 351 issues.

Table 2 shows the average level of literacy among patients admitted to the training hospitals in the present study. To this end, the literacy level of 500 patients admitted to both hospitals was recorded. The number obtained (6.72±4.34) was placed in grades six and seven i.e. elementary school according to readability indexes.

 

Table 2: Average level of literacy among patients admitted to Imam Ali (AS) Hospital and Imam Reza (AS) Hospital in the city of Bojnourd

number

mean±SD

max

min

500 individuals

6.42±4.34

18

0

 

In Table 3, the average level of readability for all pamphlets in the present study was determined based on the techniques of readability level. According to the guides, each number was interpreted and it was revealed that the majority of pamphlets were at college level. It should be noted that the number resulted from the formula specifies that the contents of the pamphlets or manuscripts are appropriate for which formal education classes in terms of level of readability. In this respect; Grades of Gunning Fog Index, Flesch-Kincaid Grade Level Formula, McLaughlin’s SMOG Readability Formula, and Powers-Sumner-Kearl Readability Formula were equivalent to formal classes in Iran. The level of Fry Readability Index is determined based on a graph which is the same as that by Gunning Fog. Categories in Flesch Grade Level Readability Formula are quite different and as the number in formula operation becomes greater, its level gets easier.

 

Table 3: Level of readability for pamphlets based on readability indexes

method of analysis

McLaughlin (SMOG) Readability Formula

Flesch Grade Level Readability Formula

Flesch-Kincaid Grade Level Formula

Gunning Fog Index

Powers-Sumner-Kearl Readability Formula

Fry’s Graph  Readability Index

PEMAT

mean±SD

13.58±1.1

12.71±24.1

14.35±3.2

19.71±2.7

10.69±1.2

16.7±3.2

12.6±4.5

interpretation of average level of readability

college education

late college level

college education

college graduate

high school

college education

grade 4

(it cannot be used)

N = 69

 

According to Table 4, 100% of pamphlets were at college level based on Flesch-Kincaid Grade Level Formula and Powers-Sumner-Kearl Readability Formula. In Flesch Grade Level Readability Formula, majority of pamphlets (79.7%) were at late college level and written at a very difficult level. According to McLaughlin’s SMOG Readability Formula, the bulk of pamphlets (91.3%) were placed at college level. Based on the PEMAT checklist, most of pamphlets (43.5%) scored below 13 i.e. grade 4.

As seen in Table 5, there was a considerable difference between the readability level of pamphlets before and after corrections and revisions according to existing standards based on McLaughlin’s SMOG Readability Formula and PEMAT indexes which was significant based on the results of paired t-test (p<0/05). McLaughlin’s SMOG Readability Formula showed that the majority (23.3%) of pamphlets were placed in grade seven after corrections and revisions. Furthermore, based on the results of the PEMAT checklist, the bulk of pamphlets (86.9%) were scored 18 to 20 i.e. grade one following the corrections and revisions to the pamphlets.

 

Table 4: Comparison of relative and absolute frequency of pamphlets based on readability levels in terms of individual readability indexes

Index

Gunning Fog Index

Fry’s Graph Readability Index

Powers-Sumner-Kearl Readability Formula

 

Flesch Grade Level Readability Formula

McLaughlin (SMOG) Readability Formula

Flesch Grade Level Readability Formula

PEMAT

readability level

number (percentage)

number (percentage)

number (percentage)

 

number (percentage)

number (percentage)

number (percentage)

number (percentag)

1-6 first to sixth grades

 

7 seventh grade

 

8 eight grade

 

9 ninth grade

 

10 start of high

school

 

11 high school

 

12 end of high school

 

higher than 12

college

 

 

-

 

 

-

 

-

 

-

 

-

 

 

-

 

-

 

 

69 (100)

-

 

 

-

 

-

 

-

 

-

 

 

-

 

-

 

 

69 (100)

-

 

 

-

 

-

 

-

 

-

 

 

-

 

-

 

 

69 (100)

 

1 (1.14)

 

 

-

 

1 (1.14)

 

1 (1.14)

 

6 (8.7)

 

 

7 (10.1)

 

53 (76.8)

 

 

-

-

 

 

-

 

-

 

-

 

-

 

 

2 (2.9)

 

4 (5.8)

 

 

63 (913)

 

 

 

90-100          fifth grade                            very easy

 

80-90            sixth grade                           easy

 

70-80            seventh grade                      relatively easy

 

60-70            eight to ninth grades           normal

 

50-60            tenth to eleventh grades      relatively difficult

 

30-50            early university                   difficult

 

0-30              late university                     very difficult

 

 

1 (1.4)

 

-

 

-

 

-

 

1 (1.4)

 

12 (17.4)

55 (79.7)

 

 

18-20                 grade 1                          It can be used unchanged

16-17                 grade 2                          It can be used following minor corrections and revisions

13-15              grade 3                         It can be used following major corrections and revisions

below 13            grade 4                          It cannot be used

 6 (8.7)

6 (8.7)

27 (39.1)

30 (43.5)

                   

 

 

 

Table 5: Comparison of the readability level of pamphlets before and after their corrections and revisions based on selected indexes

method of analysis

mean±SD

before correction and revision

mean±SD

after correction and revision

p-value

McLaughlin’s SMOG Readability Formula

13.58±1.1

8.80±1.40

0.001

PEMAT checklist

12.65±4.5

18.9±2.3

0.001

 

 

Discussion

In the present study, the readability level of preliminary pamphlets was written in very complex manner. Based on these findings, the average level of literacy among 500 patients in two training hospitals in the city Bojnurd was at grades six and seven while most pamphlets were at college level of literacy based on readability indexes. According to Gunning Fog Index, Flesch-Kincaid Grade Level Formula, and Powers-Sumner-Kearl Readability Formula; all the pamphlets were at college level. According to Powers-Sumner-Kearl Readability Formula, majority of pamphlets were written in late-college and very difficult levels. Given the results of McLaughlin’s SMOG Readability Formula, the bulk of pamphlets were at college level and majority of pamphlets scored lower than 13 i.e. grade four based on the PEMAT checklist. Such level of difficulty was also evident in studies conducted internationally. Spadero (1983) in a study evaluated 55 patient education pamphlets and found only 14 cases (25.5%) at readability level of 9 (24). Evaluating the level of readability for 190 patient education pamphlets in the United States and Canada, Lapierre and Mallet (1987) obtained an average level of 11.5±1.7 (25). Doak and Doak also obtained 7 for the literacy level of the population in their study while 10 was acquired on 100 selected sample of pamphlets which indicated a difference between the readability level of pamphlets in their statistical population and given standards (26).

All patient education programs are implemented since it is assumed that patients need to know and understand something in order to do learn and perform it.  Therefore, an effective patient education requires precise and calculated planning. The first stage in patient education planning is assessment of patients’ needs and abilities in a statistical population whose identification is an index for planning in a society. Under these conditions, it can be assured that patients seeking for education are motivated to change their behaviors and the educational programs have been effective. According to the related literature, patient education pamphlets associated with health education are often written in a very difficult level and they need one index in the study population to be changed. Although the basic indices have been identified in numerous research studies, any defined indices are based on the literacy level in the context in which the research has been conducted and it cannot be used as a fixed number in another population especially with a different language. Davis and others obtained 11 t0 14 for the readability level of patient education pamphlets in 1990, while the average level of readability within 5 months of admission to a hospital was 6 in that community. With a clear difference between the ability to read among people in the study and the level of educational materials, the researchers were required to make the readability level to the literacy level in society closer and finally the average level of readability for pamphlets reached 7 after corrections and revisions (27). In another study, Spadero (1983) estimated the average level of readability for pamphlets below 9 (24). Also, Lapierre and Mallet (1987) reported the average level of readability for pamphlets 8 or lower (25) and Farell, Mill, and Gontry (1989) reported the average level of readability below 8 (28). Matthews and Thornton (1985) also registered 6.5 to 8.5 for the average level of readability of pamphlets (29).

 

Considering that all the related studies in this field had been conducted abroad and due to the proposed variable numbers in different investigations, initially the average level of literacy among 500 hospitalized patients in a statistical population was obtained (6.72±4.34). Based on the interpretation of the above readability indexes, literacy level of the population studied was elementary schooling or a number between 6 and 7 which was consistent with the results of studies by Davis, Spadero, Lapierre and Mallet; Farell, Mill, Gontry; and Matthews and Thornton (24, 25, 27, 29). In this study, all the pamphlets were evaluated quantitatively based on six readability indexes as well as the PEMAT checklist. The average level of readability for pamphlets indicated that most of these pamphlets were written at college level and they were higher than the literacy level in the statistical population.

After correcting and revising the pamphlets, the average level of readability for pamphlets turned into 8.08±1.40 according to McLaughlin’s SMOG Readability Formula which revealed that the readability level had become closer to the literacy level in the population and significantly improved. Moreover, based on the results of the PEMAT checklist, the average readability level of pamphlets was 18.9±2.3 and it had reached to grade one from grade four after corrections and revisions showing that the pamphlets could be used unchanged. Such changes demonstrated improved quality of pamphlets in the given population in the present study.

 

Implications for Practice

Content analysis of patient education pamphlets not only contributes to their improved efficiency and effectiveness but also leads to accomplishment of desired self-care goals. For this purpose, recognizing the weaknesses and strengths of patient education pamphlets and their contents based on scientific and experimental (not mental) indices and improvement of the re-design of their structure can have a basic role in the systematic process of patient education and facilitate learning for patients. In order to fulfill the above indices and given the novelty of the study in Iran, it is expected that the results of this study help researchers of patient education in an effective manner.

Furthermore; considering that the study was conducted for the first time in Iran and researchers did not have any indexes to compare the readability level of pamphlets and the average level of literacy in society, the literacy was determined over a year on a population of 500 patients. Given the novelty of the present study, it is suggested that researchers conduct similar studies in different cities in order to obtain an accurate number for the average level of literacy in society at a national scope.

 

Acknowledgments

This study was a part of a research project approved with the code (no-93.41.1773) in the Faculty of Nursing and Midwifery in the city of Bojnurd (in northeastern Iran). We hereby appreciate all the managers and nursing staff in Imam Reza (AS) Hospital and Imam Ali (AS) Hospital affiliated to Bojnurd University of Medical Sciences for their cooperation in the fulfillment of this project.

 

Conflict of interest

The authors declare that there is no conflict of interest

  1. Williams S.A., Schreier A.M. The Role of Education in Managing Fatigue, Anxiety, and Sleep Disorders in Women Undergoing Chemotherapy for Breast Cancer. Appl Nurs Res. 2005; 18 (3): 138-47.
  2. Krupat E., Fancey M., Cleary P.D. Information and Its Impact on Satisfaction among Surgical Patients. Soc Sci Med. 2000; 51 (12): 1817-25.
  3. Estrada C.A., Hryniewicz M.M., Higgs V.B., Collins C., Byrd J.C. Anticoagulant Patient Information Material Is Written at High Readability Levels. Stroke. 2000; 31 (12): 2966-70.
  4. Treacy J. T., Mayer Deborah K. Perspectives on Cancer Patient Education. Seminars in oncology nursing; Elsevier: 2000.
  5. Sumner W. An Evaluation of Preventive Health Messages Readable. Fam Med. 1991; 23 (6): 463-6.
  6. EH Winslow. Patient Education Materials: Can Patients Read Them, or Are They Ending up in the Trash? ‎Am. J. Nurs. 2001; 101 (10): 33-8.
  7. Aldridge M.D. Writing and Designing Readable Patient Education Materials. Nephrol Nurs J. 2004; 31 (4): 373-7.
  8. Fry Readability formula. Http://Www.readabilityformulas.com/Fry-Graph-Readability-Formula.php . Accessed January 2015.
  9. Gunning Fog Index Readability Formula, or Simply Called FOG Index. Http://Www.readabilityformulas.com/Gunning-Fog-Readability-Formula.php. Accessed January 2015.
  10. The Flesch Grade Level Readability Formula. Http://Www.readabilityformulas.com/Flesch-Grade-Level-Readability-Formula.php. Accessed January 2015.
  11. Kincaid J.P., Braby R., Mears J.E.. Electronic Authoring and Delivery of Technical Information. Journal of instructional development. 1988; 11 (2): 8-13.
  12. Alexander R.E. Readability of Published Dental Educational Materials. J Am Dent Assoc. 2000; 131 (7): 937-42.
  13. Hedman A.S. Using the SMOG Formula to Revise a Health-Related Document. Am J Health Educ. 2008; 39 (1): 61-4.
  14. McLaughlin G.H. SMOG Grading: A New Readability Formula. Journal of reading. 1969; 12 (8): 639-46.
  15. Fazlollahi S, Maleki Tavana M. Methodology of Content Analysis Techniques with Emphasis on Text Readability Assessment and Determination of the Coefficient of Conflict. Pazhuhesh. 2010; 2(1):71-94. (Persian)
  16. Fazlollahi S, Maleki Tavana M. The Evaluation and Assessment of Maleki-Tavana Readability of the Third Grade in Elementary School Science Book according to 5 Standard Formulas: Fry, Gunning, Flesch, Laughlin and Close. J Curriculum Stud. 2011; 6 (22): 141-62. (Persian)
  17. Shoemaker S.J, Wolf M.S, Brach C. Development of the Patient Education Materials Assessment Tool (PEMAT): a New Measure of Understandability and Actionability for Print and Audiovisual Patient Information. Patient Educ Couns. 2014; 96(3):395-403.
  18. Badarudeen S., Sabharwal S. Assessing Readability of patient Education Materials: Current Role in Orthopaedics. Clin Orthop Relat Res. 2010; 468(10):2572-80.
  19. Redman B.K. Measurement Tools in Patient Education. 2nd Ed. New York: Springer Publishing Co. 2003.
  20. Falconer N, Reicherter E, Billek-Sawhney B, Chesbro S. An Analysis of the Readability of Educational Materials on the Webpage of a Health Consumer Organization Prosfessional: Considerations for Practice. J Allied Health. 2011; 9: 1-9.
  21. National Library of Medicine, National Institutes for Health. How to Write Easy-to-Read Health Materials. Http: // wwwnlmnihgov / Medlineplus / Etrhtml .updated on January the 2015th .Accessed February 2013.
  22. Heydari P. The validity of some popular Heydari Readability formulas. Mediterr J Soc Sci. 2012; 3 (2): 423-35.
  23. Shekarey A, Najareyan Z. A study of the Readability of Hedyehaye Asemani Textbooks for Grades Four and Five Based on Gunning Scale. Curriculum planning Knowledge and Research. 2012; 9 (6): 73-9. (Persian)
  24. Spadero D. Assessing Readability of Patient Information Materials. Pediatr Nurs. 1982; 9 (4): 274-8.
  25. Lapierre G, Mallet L. Readability of Materials. Can Pharm J. 1987; 120: 718-28.
  26. Doak C.C., Doak L.G., Root J.H. Teaching Patients with Low Literacy Skills. Am. J. Nurs. 1996; 96 (12): 16M.
  27. Davis T.C., Crouch M.A., Wills G., Miller S., Abdehou D.M. The Gap Between Patient Reading Comprehension and the Readability of Patient Education Materials. J Fam Pract. 1990; 31 (5): 533-8.
  28. Farrell-Miller P., Gentry P. How Effective Are Your Professional Development Patient Education Materials? Guidelines for Developing and Evaluating Written Educational Materials. Diabetes Educ. 1989; 15 (5): 418-22.
  29. Matthews P.J., Thornton L., McLean L. The use of Readability Fomlulas in Patient Education Materials. Paper presented at the American Association for Respiratory Therapy Summer Forum, Reno, NV. 1985.