Development and Psychometric Evaluation of Speech and Language Pathology Evidence-Based Practice Questionnaire (SLP-EBPQ)

Document Type: Original Quantitative and Qualitative Research Paper

Authors

1 Assistant Professor of Speech and Language Pathology, Neuromuscular Rehabilitation Research Center, Semnan University of Medical Sciences, Semnan, Iran

2 Assistant Professor of Speech and Language Pathology, Department of Speech and Language Pathology, Rehabilitation Research Center, School of Rehabilitation Sciences, Iran University of Medical Sciences, Tehran, Iran

3 Professor of Health Education, Department of Basic Sciences in Rehabilitation, School of Rehabilitation Sciences, Iran University of Medical Sciences, Tehran, Iran

4 Professor in Nursing Education, Behavioral Sciences Research Center, Life Style Institute, Nursing Faculty, Baqiyatallah University of Medical Sciences, Tehran, Iran

5 Assistant Professor of Occupational Therapy, Department of Occupational Therapy, School of Rehabilitation Sciences, Iran University of Medical Sciences, Tehran, Iran

6 PhD Candidate in Speech and Language Pathology, Department of Speech and Language Pathology, School of Rehabilitation Sciences, Iran University of Medical Sciences, Tehran, Iran

Abstract

Background: To date, there is no specific instrument to measure evidence-based practice (EBP) in Speech and Language Pathology (SLP). Therefore, it is essential to design a valid and reliable instrument in the EBP field for SLP.
Aim: To develop a speech and language pathology evidence-based practice questionnaire (SLP-EBPQ) for the Iranian context and evaluate its psychometric properties.
Method: This study was performed in two stages, first development of the instrument based on the literature review and semi-structured interviews with 14 speech and language pathologists and second the evaluation of the psychometric properties. Content validity of the instrument was assessed by SLP experts who were experienced in the field of EBP. Furthermore, exploratory factor analysis (EFA) and comparison of the recognized groups were conducted to determine the initial construct validity of the SLP-EBPQ. The reliability of the questionnaire was determined using internal consistency and test-retest reliability. A total of 280 speech and language pathologists completed SLP-EBPQ to evaluate construct validity and internal consistency. Furthermore, 30 speech and language pathologists completed the SLP-EBPQ after 2 weeks for test-retest reliability.
Results: The developed instrument was a questionnaire with 77 items. The results of EFA demonstrated that the SLP-EBPQ contained nine factors with appropriate internal consistency (α=0.635-0.885). Moreover, the Intra-class Correlation coefficient of the factors was (r=0.814-0.966) in the test-retest reliability.
Implications for Practice: The SLP-EBPQ is a valid and reliable instrument and can be applied to evaluate EBP among speech and language pathologists for educational, clinical, or research purposes.

Keywords

Main Subjects


Introduction

Evidence-Based Practice (EBP) is a process seeking to improve the provision of services to patients by considering the best research evidence, clinical experiences of therapists, and patient’s needs and preferences (1, 2). This systematic process helps therapists to make better clinical decisions. Despite its root in medicine, application of EBP in other fields related to health sciences has extensively increased all over the world (3-9). In addition, teaching EBP is a part of the curricula in many disciplines (10).

Today, EBP is the center of attention due to its benefits, including more effective provision of services for each individual, more accountability toward patients and their families, creation of equal treatment for clients according to scientific evidences, updating of the knowledge of the therapists, improvement in the quality of service provision, increase of the professional credibility, encouragement of adherence to ethical principles, improvement in interdisciplinary collaborations and communications, and encouragement of continuing education (5, 11-13).

As a member of the healthcare team, speech and language pathologists are required to pay more attention to EBP and make their clinical decisions based on evidence (2, 14) (15). The evaluation and review of the activities in various countries to extend the use of EBP in Speech and Language Pathology (SLP) highlight the importance of this issue (3). Scientific associations affiliated to SLP, such as Royal College of Speech and Language Therapists and American Speech-Language-Hearing Association, have also emphasized the use of EBP in SLP (16). Moreover, these associations have defined research and development in EBP as their priorities (4, 16, 17). The evaluation of the current status and documentation of the performance of speech and language pathologists about EBP could be an important step for the greater use of EBP. Accordingly, there is need to design an instrument, which can assess the status of EBP in SLP.

Today, there is a variety of questionnaires to evaluate EBP in various fields of medicine (7, 18-21). However, to our knowledge, there is no comprehensive instrument for assessment of EBP, which is specifically designed for speech and language pathologists. Meanwhile, each field or profession has its own properties, which can affect the use of EBP (22). As a result, a special SLP questionnaire must be designed in related to EBP. With this background in mind, this study aimed to develop and evaluate the psychometric properties of the Speech and Language Pathology-Evidence-Based Practice Questionnaire (SLP-EBPQ).

 

Methods

This methodological research was performed in two stages: 1) design of SLP-EBPQ through the review of the literature and semi-structured interviews with a group of speech and language pathologists (n=14), and 2) evaluation of psychometric properties of SLP-EBPQ by assessing its reliability and validity.

At first, the first author searched the Scopus, PubMed (Medline), and Web of Science databases as well as Google Scholar search engine for the published articles up to April 2017 in order to extract the existing questionnaires and instruments in the EBP field. Various keywords were applied to the comprehensive assessment of the present questionnaires. These keywords included ‘evidence-based practice’ in combination with ‘profile’, ‘tool’, ‘questionnaire’, ‘assessment’ and ‘instrument’. Moreover, manual search in the EBP-related articles published in well-known journals, was carried out to ensure the access to the maximum coverage of the information in this area. The inclusion criteria were: All English or Persian articles related to EBP, and access to the full text of articles. Non-English, non-Persian articles, and articles with no available questionnaire were excluded from the study. As a result, 17 articles were found and used for the item generation stage. This literature review resulted in 307 items.

Qualitative research was conducted to compile the experiences of speech and language pathologists toward EBP. In this section of the study, semi-structured interviews were conducted with 14 speech and language pathologists selected through purposive sampling. The reason for this was to achieve maximum variation in terms of years of SLP experience, education degree, workplace, clinical practice field, and types of roles.

The main questions of the interviews were 1) could you please describe your experience with EBP in clinical works? 2) What kind of EBP activities (e.g., participating in a journal club, reading articles, attending workshops and congresses, participating in continuing education programs) do you participate in? 3) What barriers do you faced in the realm of EBP? 4) What factors have you experienced as facilitators for the application of EBP? 5) Could you please explain about your experience in the application of EBP? And 6) what is the outcome of the application of the EBP in clinical practice? When data saturation was reached, the data collection stopped.

Interviews were analyzed applying conventional content analysis approach (23). To this end, the following steps were conducted: (1) transcribing the interviews, (2) deriving codes by reading the interviews word by word, (3) sorting the derived codes into categories, and (4) sorting the categories to create themes (24). Consequently, the primary questionnaire was designed based on the concept explained in the qualitative section of the study and literature review.

In total, 447 items were generated after the review of the literature and conducting semi-structured interviews. These items were evaluated by the research team and experts in several sessions to eliminate duplicated items (201 items) and merge items with similar meaning (135 items). with 111 items on five main domains, including attitude toward EBP, knowledge about EBP, use of EBP, barriers to the use of EBP and facilitators of EBP application.

The face validity of the SLP-EBPQ was determined qualitatively and quantitatively. At first, face to face interviews were conducted with 10 speech and language pathologists, and they were asked to study the questionnaire items and provide insights about the ‘relevancy’, ‘ambiguity’ and ‘difficulty’ of the items. Item Impact Score (IIS) was applied to determine quantitative face validity. In this regard, 10 SLP experts expressed their opinions about the importance of each item on the basis of the 5-point Likert scale ranging from the most important (score 5) to the least important (score 1). The IIS for each item is calculated as follows: Item Impact Score= percentage of raters who scored a score of 4 or 5 × mean score for the importance of each item. Each item with IIS scores of 1.5 or above was selected for the next stage, and those with lower scores were eliminated from the questionnaire (25, 26).

In the qualitative content validity assessment, 15 experts from the SLP faculty members (10 with PhD degree and 5 PhD students), who have experiences in EBP, evaluate the questionnaire items regarding grammar standards, use of proper phrases, necessity, importance and type of scoring, and the order of phrases in the accurate place (27). Quantitative content validity was determined using two indicators, namely content validity ratio (CVR) and content validity index (CVI). To determine CVR, the opinions of the 15 experts were used to provide insights on the necessity of each item according to the 3-point Likert Scale (1=essential, 2=useful but not essential, and 3=unessential). According to Lawshes’ Table and with regard to the participation of 15 experts at this stage, items with CVR scores of 0.49 or above were maintained (28). After this stage, the same 15 experts provided their opinions about the relevancy of the items according to a 4-point Likert scale (1=non-relevant, 2=fairly relevant, 3=relevant, and 4=completely relevant) to determine the CVI (29). The required amount of CVI for maintaining each item at this stage was ≥0.78 (29). Moreover, the mean of CVI was calculated according to the mean CVI score of all questionnaire items. At this level, the required amount for S-CVI/Ave was ≥0.9 (29). Furthermore, the Cohen's  kappa coefficient was used to provide the complementary information about the CVI, since it provides data about the agreement degree beyond the chance (29).

To analyze the items, the questionnaire was distributed among 43 speech and language pathologists. This stage aimed to evaluate the problems in the questionnaire, type of scoring, and required time to fill the questionnaire. Afterwards, the participants were asked to express their opinions about the clarity and comprehensiveness of the items.

Preliminary reliability is a type of item analysis with an emphasis on internal consistency. At this stage, the loop method was applied to evaluate the effect of elimination of each item on the increase of the internal consistency of that domain (Cronbach’s alpha). This stage is an essential step toward the evaluation, revision, and/or elimination of improper items before carrying out the construct validity. Furthermore, the correlation of each item with the domain total score and other items was assessed in order to eliminate items which have low correlation with the domain total score. Moreover, discrimination and difficulty index was calculated for the items of the knowledge domain of SLP-EBPQ to eliminate the items with the lack of necessary criteria.

Exploratory Factor Analysis (EFA) was used to explore the factors of the SLP-EBPQ. To this end, we used the principal components analysis with varimax rotation (considering the independence of the factors). If there was a missing data at this stage, it would be replaced by the median. Sampling adequacy to perform EFA was assessed through the Kaiser-Meyer-Olkin (KMO) and Bartlett’s Test of Sphericity. The sample size is adequate to perform EFA when the KMO value was above the 0.5 (30, 31). Eigenvalues above one and the scree plot method were used to decide the number of prominent factors (30).

The minimum factor load for maintaining each item in factors extracted from factor analysis was determined at 0.3. After this stage and extraction of SLP-EBPQ factors, the level of fitness was assessed for these factors and main dimensions of SLP-EBPQ. Furthermore, the comparison of the recognized groups was performed to explore the construct validity and determine the ability of the SLP-EBPQ to properly separate different groups. To this end, the educational level of the participants was divided into two categories of BSc (undergraduate) and above BSc (postgraduate). Following that, independent t-test was exploited to compare the items related to the factors of SLP-EBPQ between the two groups. The skewness and kurtosis were used to investigate the normality of the data.

Internal consistency was assessed using Cronbach’s alpha coefficient, and test-retest reliability was used to evaluate the reliability of the SLP-EBPQ. A Cronbach’s alpha coefficient of 0.7 or above was considered satisfactory (32). In order to investigate test-retest reliability, the questionnaire was completed by 30 speech and language pathologists in two stages with a two-week interval. Subsequently, the obtained scored in the two stages were compared using the intraclass correlation coefficient (ICC) with a two-way mixed model and absolute consistency type. The test-retest reliability considered excellent if the ICC was 0.75 or above (33).

To evaluate the SLP-EBPQ, the speech and language pathologists from the Iranian scientific speech therapy association were selected through random sampling technique. Inclusion criteria for the participants were: (1) a bachelor or higher degree in SLP and (2) an interest to participate in the study. The exclusion criteria included: (1) incomplete consent form, (2) partial completion of the questionnaire, and (3) no clinical practice in the last three months. Three hundred questionnaires were distributed among speech and language pathologists with the mentioned criteria to evaluate the psychometric properties of the SLP-EBPQ. To determine test-retest reliability, 30 participants completed the SLP-EBPQ again with an interval of 2 weeks based on their own interest.

All statistical analyses performed using SPSS software (version 18.0; SPSS, Inc., Chicago, IL). Descriptive statistics including frequencies, means, and standard deviations were used to conduct both item- and subscale-level analyses. P-value less than 0.05 considered statistically significant.

It should be noted that this research was approved by the Ethics Committee affiliated with Iran University of Medical Sciences, Tehran, Iran (IR.IUMS.RE1395.9221363202). In addition, the objectives of the research were explained to the participants and they were assured of the confidentiality regarding their personal information. Furthermore, participation in this study was voluntary and the subjects were allowed to withdraw from the study at any time. Finally, written informed consent forms were obtained from the participants who agreed to participate voluntarily in this study prior to the research.

 

Results

From 300 questionnaires distributed among the speech and language pathologists, 290 were returned. In addition, 10 questionnaires were eliminated due to incomplete information, and a total of 280 questionnaires were evaluated in terms of psychometric properties. Demographic characteristics of the participants are shown in Table 1.

The obtained results of IIS revealed that the face validity of the items was within the range of 3.4-5. Given the fact that the IIS of all items was above 1.5, none of the items were eliminated at this stage. Furthermore, some changes were applied to the items of SLP-EBPQ in qualitative assessment of content validity according to the opinions of 10 speech and language pathologists in order to ensure the comprehension of the items.

In the qualitative evaluation of content validity, some changes were applied to the items according to the opinions of the experts. Furthermore, several items were merged due to their similarity. Therefore, the number of items decreased to 107 ones.

In the quantitative evaluation of content validity, 25 items were eliminated due to CVR scores below 0.49. Nevertheless, 19 items were removed according to the opinion of the research team. Meanwhile,

Table 1. Demographic characteristics of participants

Variable

 

N (%)

Gender

Female

197 (70.4)

Male

83 (29.6)

Age

Mean (SD)= 28.7 (5.7)

 

Degree

Bachelor

125 (44.6)

Master student

44 (15.7)

Master

70 (25)

PhD student

35 (12.5)

PhD

6 (2.1)

Workplace

Private

101 (36.1)

Public

49 (17.5)

Both

130 (46.4)

Experience

<3

109 (38.9)

3-5

134 (47.9)

>5

37 (13.2)

Mean (SD)

5.7 (5.1)

Total

 

280

 

six of these items were kept due to their high importance. No item was eliminated in the CVI stage. Moreover, the questionnaire scale-level content validity index (S-CVI) was 0.952 and modified kappa coefficient was 0.79-1, which was desirable (29). After the evaluation of face and content validities, an 88-item questionnaire was prepared for the next stage.

According to the results of the pilot and item analysis stages, another eight items were eliminated at this stage. Moreover, two items from the attitude section, two items from the barriers section and one item from the section of facilitators were eliminated due to their negative impact on the Cronbach’s alpha and lack of significant correlation with other items. On the other hand, three items were eliminated from the knowledge section of the questionnaire with regard to the discrimination and difficulty index.

In the current study, EFA was performed on 80 items and the KMO coefficient was 0.79. Moreover, the value of Bartlett’s Test of Sphericity was estimated at 9937.550, which was significant at P

An oblique factor rotation detected nine latent factors. The extraction was according to scree plot visual interpretation and Kaiser’s criterion for Eigenvalues of equal to or greater than unity. The nine factors, comprising 77 of the original 80 items, explained 46.92% of the total variance. Three items were removed from the questionnaire due to their low loading on the factors. In total, nine factors were identified according to the results of factor analysis and at the end of this stage, 77 items remained in the questionnaire (Figure 1). Information related to the factors, titles, the number of items, and their percentage of variance is presented in Table 2.

According to the results of the evaluation of construct validity, individuals with an educational level above BSc had a greater attitude and more knowledge and usage of external evidence, compared to individuals with BSc. This difference between the groups was statistically significant. In addition, individuals with BSc faced more barriers while using EBP, compared to those with degrees above BSc. This difference was statistically significant, with the exception of a factor of individual obstacles. Moreover, no significant difference was observed between the groups in terms of facilitators to use of EBP (Table 3). It should be noted that the values of the skewness and kurtosis were +1.7 and +2.1, respectively.

The values of Cronbach’s alpha of various factors in SLP-EBPQ were determined within the range of 0.635-0.885 (Table 4). In addition, the results of the test-retest reliability of various factors in SLP-EBPQ were within the range of 0.814-0.966 by the evaluation of ICC, which was indicative of the proper test-retest reliability of the SLP-EBPQ. Details related to the values of ICC of various factors of SLP-EBPQ are presented in Table 4.


 

Table 2. Factors, items, and factor loadings for Speech-Language Pathology Evidence-Based Practice Questionnaire (n=280)

Domains Cumulative

%=46.92

Item

Factor 1

Factor 2

Factor 3

Factor 4

Factor 5

Factor 6

Factor 7

Factor 8

Factor 9

Attitude

% of variance=7.22

I need to use evidence-based treatments.

0.775

 

 

 

 

 

 

 

 

The use of EBP in speech and language pathology is essential.

0.741

 

 

The EBP helps me to clinical decisions making.

0.722

 

The EBP is a waste of time.

0.707

 

I am ready to try using EBP in my work.

0.674

 

I am interested in using EBP in my clinical practice.

0.653

 

I would like to receive training courses on EBP and necessary skills for its implementation.

0.636

 

Scientific texts and research findings are helpful in my daily clinical practice.

0.611

 

The EBP is the basis of professional performance.

0.609

 

It is important to use the results of the previous research findings so that I have a timeline for doing it in my schedule.

0.553

0.358

The EBP is a transitory fashion mode and disappears over time.

0.487

 

Facilitator

% of variance=13.25

The presence of enough space to discuss scientific evidence with colleagues at work

 

0.800

 

 

 

 

 

 

 

The presences of scientific meetings with colleagues

0.795

The presences of workshops

0.774

The availability of the appropriate facilities at work for using EBP

0.734

The presences of specialized conferences

0.728

Prioritizing the use of the scientific evidence results at work

0.708

The presence of expert’s knowledge and experience

0.602

Supporting my colleagues at work to use EBP

0.571

The presence of the cyberspace to connect with colleagues and share experiences with them

0.468

 

Table 2 Continued.

Knowledge

% of variance=19.16

What is a study that uses statistical methods to analyze data of the several studies?

 

 

0.603

 

 

 

 

 

 

What is it called? “Generalizability of the research results from the sample to the target population”

 

0.548

Which of the following studies has a higher level of evidence than others?

 

0.529

Which of the abilities do not require in order implementing EBP?

 

0.509

With considering the 95% probability, what is the amount of the P-Value is needed for significant statistical results?

 

0.495

What is a kind of medical studies aimed to reduce potential bias in assessing the effectiveness of a new treatment?

 

0.483

Which of the following items do not a part of the EBP process?

 

0.457

Evidence-based practice does not emphasize the following items?

 

0.454

What kind of confidence estimate is calculated from the observed evidence?

 

0.427

Which of the following items do we use to find the right keyword in order to search in the PubMed database?

0.351

0.390

Environmental Barriers

% of variance=24.98

Time-consuming of an appropriate client’s assessment

 

 

 

0.666

 

 

 

 

 

The difficulty of coordinating with the client’s needs and preferences

 

0.662

 

 

Lack of time to implement EBP

 

0.601

 

 

Lack of public awareness about EBP

 

0.565

 

0.354

Not participating family in decision making

 

0.557

 

 

Lack of facilities at work to implement EBP

 

0.539

 

 

The difficulty of the evidence-based treatment process

 

0.495

 

 

 

Table 2 Continued.

Environmental Barriers

% of variance=24.98

Lack of proper supervision of the performance of clinicians

 

 

 

0.455

 

 

 

 

 

Difficulty in the generalization of the research finding to my clients

 

0.434

0.400

 

Lack of the authority at workplace to change in clinical practice

 

0.433

 

 

Not having access to scientific colleagues to discuss research findings

 

0.405

 

 

Not accepting teamwork in clinicians’ culture

 

0.401

 

 

The absence of the native norms in speech and language pathology

0.352

0.359

 

 

The speech and language pathology’s need to communicate with many disciplines

 

0.345

 

 

Personal Barriers

% of variance=30.39

Unwillingness to change previous ideas and testing new ideas

 

 

 

 

0.745

 

 

 

 

Lack of the interest to EBP

0.741

Belief to the adequacy of the knowledge after graduation

0.657

Lack of personal motives

0.649

Not believing to research findings

0.646

An early negative judgment of the clinicians on the efficacy of the treatments for their clients

0.608

Barriers related to Skill, education, and access to evidence % of variance=35.33

No access to evidence (i.e. articles, books, guidelines)

 

 

 

 

 

0.697

 

 

 

Lack of native treatment programs or clinical guidelines

0.633

Inability to find scientific evidence

0.573

Lack of the mastery of the language of scientific evidence

0.555

Lack of knowledge about EBP

0.522

 

Table 2 Continued.

 

Difficulty in understanding the statistical analysis

 

 

 

 

 

0.502

 

 

 

Barriers related to Skill, education, and access to evidence % of variance=35.33

Not having access to the full text of articles

 

 

 

 

 

0.486

 

 

 

Lack of enough knowledge to read articles

 

 

 

 

 

0.451

 

 

 

Lack of EBP education at universities

0.445

Use of External evidence

% of variance=39.99

Case Reports articles

 

 

 

 

 

 

0.729

 

 

Randomized controlled trials or Single subject articles

0.711

Systematic review articles

0.693

Clinical guidelines

0.579

Textbooks

0.543

Internet resources

0.503

Educational video or audiotapes

0.502

works shops and continuing education programs

0.412

Workshops related Barriers

% of variance=43.48

Workshops are being very theoretical

 

 

 

 

 

 

 

0.710

 

Workshops are not coordinated with clinicians’ clinical needs

0.662

Inappropriate time for holding workshops

0.515

Conferences are not useful clinically

0.492

Use of Internal evidence

% of variance=46.92

Opinions of colleagues (speech-language therapists)

 

 

 

 

 

 

 

 

0.724

Expert consultation

0.701

Telegram groups and channels or other same social networks

0.537

Consultation with team members from other fields

0.449

Client’s needs and preferences

0.376

My own clinical experience

0.332

 

Table 3. Results of comparing scores of different factors of the Speech and Language Pathology Evidence-Based Practice Questionnaire between two groups with different degrees (n = 280)

Factor

Undergraduate (n=125), Mean (SD

Postgraduate (n=155), Mean (SD)

P-Value

Attitude

45.6 (3.8)

48.8 (4.5)

P<0.001

Knowledge

2.1 (1.8)

5.6 (2.5)

P<0.001

Use (Internal Evidence)

22.6 (2.8)

21.5 (3.2)

P<0.003

Use (External Evidence)

23.5 (5.4)

26.1 (4.7)

P<0.001

Environmental Barriers

45.9 (7.7)

43.2 (8.8)

P<0.01

Personal Barriers

13.5 (4.3)

12.8 (5.1)

P=0.25

Barriers related to skill, education, and access to evidence

30.4 (5.6)

25.5 (6.5)

P<0.001

Workshops related Barriers

14 (3)

13.2 (3)

P=0.03

Facilitator

36.9 (6.2)

35.7 (6.1)

P=0.1

 

Table 4. Cronbach's Alpha and Intraclass correlation coefficient for domains of the Speech and Language Pathology Evidence-Based Practice Questionnaire

Factor

Mean (SD)

Cronbach's Alpha

(n=280)

ICC

(n=30)

Confidence interval

(Lower- Higher)

P-Value

Attitude

49.1 (3.4)

0.885

0.871

0.938-0.732

P<0.001

Knowledge

4.6 (2.5)

0.794

0.966

0.985-0.916

P<0.001

Use (Internal Evidence)

23.1 (3.2)

0.635

0.885

0.957-0.563

P<0.001

Use (External Evidence)

25.6 (5.6)

0.767

0.943

0.982-0.586

P<0.001

Environmental Barriers

41.2 (6.2)

0.842

0.854

0.931-0.695

P<0.001

Personal Barriers

16.6 (5.5)

0.836

0.923

0.964-0.836

P<0.001

Barriers related to skill, education, and access to evidence

24.2 (5.3)

0.830

0.861

0.934-0.707

P<0.001

Workshops related Barriers

13.5 (2.3)

0.671

0.815

0.912-0.614

P<0.001

Facilitator

36.5 (5.8)

0.885

0.814

0.911-0.614

P<0.001

ICC: intraclass correlation coefficient

 

Discussion

The SLP-EBPQ was designed according to the concepts determined in the qualitative section of the research. This questionnaire is designed for evaluation of the attitude, knowledge, and use of the EBP by speech and language pathologists and assessment of the associated barriers and facilitators of EBP. Therefore, SLP-EBPQ can be used by speech and language pathologists of various countries by considering the translation of the target language, modifications based on cultural factors, and facilities of the selected country, as well as the evaluation of the psychometric properties. In this research, face and content validities (qualitative and quantitative), construct validity (exploratory factor analysis and comparison of the recognized groups), internal consistency (Cronbach’s alpha) and test-retest reliability of the questionnaire were confirmed.

In the evaluation of the content and face validities, the improper items were eliminated from the questionnaire based on the opinion of experts and respondents, and accordingly the total number of 111 items reduced to 88 items. In addition, eight items were removed in the pilot and item analysis stage. In the end, an 80-item questionnaire with five domains was entered into the stage of construct validity evaluation.

In the development of the primary questionnaire, the items of SLP-EBPQ were categorized into five domains, including attitude, knowledge, use, barriers, and facilitators. Factor analysis results demonstrated that the items of SLP-EBPQ were in nine factors. Items in the sections of attitude, knowledge, and facilitators were each placed in a separate factor, similar to the primary questionnaire. The primary questionnaire was turned into two factors of the application of internal and external evidence. Moreover, the barrier section of the primary questionnaire was changed into four various factors (environmental, personal, barriers related to skill, education and access to evidence, and workshops and congresses barriers). Given the consistency of items in the extracted factors in factor analysis with the determined dimensions of the questionnaire in the first stage, the construct validity of SLP-EBPQ was confirmed.

In the present study, the analysis of construct validity revealed that SLP-EBPQ had an appropriate level of construct validity. Moreover, comparison of participants with BSc degree and those with degrees higher than BSc revealed a significant difference between the two groups in terms of attitude, knowledge, use of evidence, and parts of the barrier. Furthermore, the obtained results of internal consistency of the questionnaire indicated that Cronbach’s alpha values of seven out of nine factors were above 0.7, which is an appropriate value. However, the Cronbach’s alpha for two other factors of the SLP-EBPQ were within the range of 0.6-0.7, which was lower than other sections. The lower values of Cronbach’s alpha in these two factors could be related to their low number of items. In the current research, the reliability of SLP-EBPQ was evaluated through test-retest. Results obtained by two implementations of SLP-EBPQ with a two-week interval demonstrated the high stability of the SLP-EBPQ in short-term.

Compared to other questionnaires and instruments used in the EBP area for various professions, SLP-EBPQ is a self-report tool similar to other questionnaires (7, 18, 19, 21, 34, 35). In addition, SLP-EBPQ contains nine domains (attitude, knowledge, use of external evidence, use of internal evidence, environmental barrier, personal barrier, barriers related to skill, education and access to evidence, workshop and congresses barrier and facilitators), the majority of them which is similar to other questionnaires available in this field.

However, in the knowledge section, in which the knowledge of person about EBP and the associated factors are evaluated, some multiple-choice questions are used to accurately and properly assess the knowledge of individuals. Meanwhile, the knowledge of person about EBP was evaluated in previous questionnaires through self-report and self-assessment, which might have had biases (19). In addition, given the fact that qualitative studies and interviews with speech and language pathologists were used to design SLP-EBPQ, the barrier section is more comprehensive and contains 33 items placed in four factors. According to our knowledge, the items for evaluation of the barriers has never been observed in other questionnaires (18, 19), and this comprehensiveness could be the strength of the SLP-EBPQ.

The questionnaire contains 77 items that makes it a comprehensive tool on EBP. However, it is essential to increase the necessary time for completing the questionnaire, which might lead to tiresome and hasty responses of the respondents. However, given the evaluation of numerous variables by SLP-EBPQ, a decrease in items might lead to reduced content validity (19).

One of the major limitations of this study was the long time required to complete the questionnaire by the participants in various stages of validity due to the large number of items. Moreover, given the lack of similar questionnaires in EBP for Iranian speech and language pathologists, we failed to assess the concurrent validity of the questionnaire. Furthermore, similar to many questionnaires in the EBP area, SLP-EBPQ is also a self-reported questionnaire and its responses might be affected by social desirability. Despite these limitations, diversity in the domains and comprehensiveness of SLP-EBPQ, proper face and content validities, acceptable internal consistency, and high reliability are the strengths of the questionnaire.

 

Implications for Practice

According to the results of the current study, SLP-EBPQ can be used to evaluate knowledge, attitude, and application of EBP of speech and language pathologists and could be beneficial for the evaluation of barriers and facilitators in this regard. The SLP-EBPQ can be used for educational, clinical, and research purposes due to its acceptable level of reliability and validity and can be a beneficial instrument in EBP area. However, it is recommended to conduct further studies to evaluate the SLP-EBPQ since it is a new questionnaire.

 

Acknowledgments

This article was extracted from the PhD dissertation of the author in speech and language pathology supported by Iran University of Medical Sciences, Tehran, Iran. We would like to thank the Research Deputy of Iran University of Medical Sciences, experts, and speech and language pathologists who cooperated with this research.

 

Conflicts of Interest

The authors declare that they have no conflicts of interest.

  1. Greenhalgh T. How to read a paper: the basics of evidence-based medicine. New Jersey: John Wiley & Sons; 2014.
  2. McCurtin A, Roddam H. Evidence-based practice: SLTs under siege or opportunity for growth? The use and nature of research evidence in the profession. Int J Lan Commun Disord. 2012;47(1):11-26.
  3. Lass NJ, Pannbacker M. The application of evidence-based practice to nonspeech oral motor treatments. Lang Speech Hear Serv Sch. 2008;39(3):408-21.
  4. Onslow M. Stuttering and its treatment: eleven lectures. Sydney: Australian Stuttering Research Centre; 2016.
  5. Brackenbury T, Burroughs E, Hewitt LE. A qualitative examination of current guidelines for evidence-based practice in child language intervention. Lang Speech Hear Serv Sch. 2008;39(1):78-88.
  6. Salehi S, Mohmedie Karbalaie A, Abedi H. A study of the implementation rate of evidence-based nursing cares by nurses in state hospitals in Ahwaz in 2011. Evid Based Care J. 2013;3(2):7-16 (Persian).
  7. Buchanan H, Jelsma J, Siegfried N. Measuring evidence-based practice knowledge and skills in occupational therapy-a brief instrument. BMC Med Educ. 2015;15(1):191.
  8. Campbell WN, Douglas NF. Supporting evidence-based practice in speech-language pathology: a review of implementation strategies for promoting health professional behavior change. Evid Based Commun Assess Intervene. 2017;11(3-4):72-81.
  9. Dang D, Dearholt SL. Johns Hopkins nursing evidence-based practice: model and guidelines. Indianapolis: Sigma Theta Tau; 2017.
  10. Spek B, Wieringa‐de Waard M, Lucas C, van Dijk N. Teaching evidence-based practice (EBP) to speech–language therapy students: are students competent and confident EBP users? Int J Lang Commun Disord. 2013;48(4):444-52.
  11. Walker BF, Stomski NJ, Hebert JJ, French SD. Evidence-based practice in chiropractic practice: a survey of chiropractors’ knowledge, skills, use of research literature and barriers to the use of research evidence. Complement Ther Med. 2014;22(2):286-95.
  12. Azizi M, Sirati Nayer M, Mokhtari Nouri J, Motahedeyan Tabrizi E. Designing and accrediting the evidence-based care guidelines on insomnia and constipation in psychiatric patients. Evid Based Care J. 2013;3(3):81-9 (Persian).
  13. Rezaeian A, Niknejad Jalali A, Ashrafzadeh F. An evidence based care package to improve motor skills of infants living in foster care according to integrative review approach. Evid Based Care J. 2013;3(2):73-87 (Persian).
  14. Reilly S. The challenges in making speech pathology practice evidence based. Adv Speech Lang Pathol. 2004;6(2):113-24.
  15. Dodd B. Evidence-based practice and speech-language pathology: strengths, weaknesses, opportunities and threats. Folia Phoniatr Logop. 2007;59(3):118-29.
  16. ZipoliJr RP, Kennedy M. Evidence-based practice among speech-language pathologists: attitudes, utilization, and barriers. Am J Speech Lang Pathol. 2005;14(3):208-20.
  17. O'Connor S, Pettigrew CM. The barriers perceived to prevent the successful implementation of evidence‐based practice by speech and language therapists. Int J Lang Commun Disord. 2009;44(6):1018-35.
  18. Jette DU, Bacon K, Batty C, Carlson M, Ferland A, Hemingway RD, et al. Evidence-based practice: beliefs, attitudes, knowledge, and behaviors of physical therapists. Phys Ther. 2003;83(9):786-805.
  19. Bernhardsson S, Larsson ME. Measuring evidence-based practice in physical therapy: translation, adaptation, further development, validation, and reliability test of a questionnaire. Phys Ther. 2013;93(6):819-32.
  20. Paul F, Connor L, McCabe M, Ziniel S. The development and content validity testing of the Quick-EBP-VIK: a survey instrument measuring nurses’ values, knowledge and implementation of evidence-based practice. J Nurs Educ Pract. 2016;6(5):118.
  21. Upton D, Upton P. Development of an evidence-based practice questionnaire for nurses. J Adv Nurs. 2006;53(4):454-8.
  22. de Beer J, Kalf JG. Evidence-based logopedie: logopedisch handelen gebaseerdwetenschappelijke evidentie. Houten: Bohn Stafleu van Loghum; 2003.
  23. Graneheim UH, Lundman B. Qualitative content analysis in nursing research: concepts, procedures and measures to achieve trustworthiness. Nurse Educ Today. 2004;24(2):105-12.
  24. Hsieh HF, Shannon SE. Three approaches to qualitative content analysis. Qual Health Res. 2005;15(9):1277-88.
  25. Broder HL, McGrath C, Cisneros GJ. Questionnaire development: face validity and item impact testing of the child oral health impact profile. Community Dent Oral Epidemiol. 2007;35(Suppl 1):8-19.
  26. Valizadeh Zare N, Mohammadi E, Zare K, Ehahi N. Design and psychometric evaluation of coping scale in recipients of kidney transplant. Evid Based Care J. 2017;7(3):45-53.
  27. Gungor I, Beji NK. Development and psychometric testing of the scales for measuring maternal satisfaction in normal and caesarean birth. Midwifery. 2012;28(3):348-57.
  28. Lawshe CH. A quantitative approach to content validity 1. Personnel Psychol. 1975;28(4):563-75.
  29. Polit DF, Beck CT, Owen SV. Is the CVI an acceptable indicator of content validity? Appraisal and recommendations. Res Nurs Health. 2007;30(4):459-67.
  30. Dennis C. The essentials of factor analysis. 3rd ed. London: A&C Black; 2006.
  31. Musavinasab M, Ravanipour M, Pouladi S, Motamed N. Psychometric properties of social support questionnaire to measure empowerment in elderly patients with cardiovascular disease in Iran. Evid Based Care J. 2018;8(1):55-66.
  32. Saei Ghare Naz M, Ebadi A, Darooneh T, Rashidi Fakari F, Kholosi Badr F, Ghasemi V, et al. Cross-cultural adaptation and psychometric evaluation of the Pap Smear belief questionnaire in Iranian women. Evid Based Care. 2018;8(3):27-34.
  33. Terwee CB, Bot SD, de Boer MR, van der Windt DA, Knol DL, Dekker J, et al. Quality criteria were proposed for measurement properties of health status questionnaires. J Clin Epidemiol. 2007;60(1):34-42.
  34. Johnston JM, Leung GM, Fielding R, Tin KY, Ho LM. The development and validation of a knowledge, attitude and behaviour questionnaire to assess undergraduate evidence-based practice teaching and learning. Med Educ. 2003;37(11):992-1000.
  35. Upton P, Scurlock-Evans L, Upton D. Development of the student evidence-based practice questionnaire (S-EBPQ). Nurse Educ Today. 2016;37:38-44.