M E T H O D O L O G I C A L I S S UE S I N N U R S I N G R E S E A R C H

Development of an evidence-based practice questionnaire for nurses

Dominic Upton BSc MSc PhD C Psychol FBPsS
Head of Psychology Division, University of Worcester, Worcester, UK

Penelope Upton MA MPhil

Research Associate, Psychology Department, Sheffield University, Sheffield, UK

Accepted for publication 30 May 2005

Correspondence: UPTON D. & UPTON P. (2006) Journal of Advanced Nursing 54(4), 454–458
Dominic Upton, Development of an evidence-based practice questionnaire for nurses
Division of Psychology, Aim. The aim of this paper is to report the development and validation of a self-
University College Worcester,
report measure of knowledge, practice and attitudes towards evidence-based prac-
Henwick Grove,
tice (EBP).
Worcester WR2 6AJ,
UK.
Background. Evidence-based practice has become increasingly important in health
E-mail: d.upton@worc.ac.uk care since the mid-1990s as it provides a framework for clinical problem-solving.
However, to date no means exist to quantify the extent to which barriers, such as
lack of time in the working day, lack of appropriate skills and negative attitudes,
may prevent greater uptake of EBP.
Methods. Questionnaire development was based on established psychometric
methods. Principal component factor analysis was used to uncover the underlying
dimensions of the scale. Internal consistency of the scale was assessed by Cronbach’s
a. Finally, construct validity was assessed via convergent and discriminant validity.
Results. The final questionnaire comprised three distinct scales (EBP, attitudes
towards EBP and knowledge of EBP), which had robust validity and internal reli-
ability.
Conclusion. This tool can be used to measure the implementation of EBP.

Keywords: attitudes, evidence-based practice, nurses, reliability, validity

which barriers, such as lack of time in the working day for

Introduction
anything other than a clinical workload and lack of appropri-
The current emphasis in the United Kingdom (UK) is for a ate skills, may prevent greater degrees of EBP.
National Health Service that is modern, dependable and offers One attempt to develop a formal measure of competence in
a quality-assured service. This drive for high quality care was evidence-based medicine resulted in the Fresno test (Ramos
formalized by the introduction of clinical governance, and et al. 2003). However, this test does not consider the extent
encompasses the twin concepts of clinical effectiveness and to which the evidence can and is put into action in daily
evidence-based practice (EBP). Evidence-based practice has working life, nor assess the attitude of individual healthcare
become increasingly important in health care in the UK since professionals towards EBP and its role in routine care. Yet
the mid-1990s, as it provides a framework for clinical problem- workforce attitudes are known to be an important issue in the
solving which allows practitioners to keep up-to-date with uptake of new guidelines and practices.
current best practice in their field (DoH 2000). Some studies have reported attempts at assessing attitudes or
Enthusiasm for EBP in nursing has led to a variety of reported practice either by postal questionnaire (e.g. O’Donnell
initiatives, including the introduction of centres for EBP, 2004) or qualitative interviews (e.g. Stevenson et al. 2004) in
dedicated journals, and websites. However, to date no means medical, nursing and other healthcare staff. Furthermore, a
exist to quantify this enthusiasm or measure the extent to measure has been developed that explores attitudes, know-

454
2006 The Authors. Journal compilation
2006 Blackwell Publishing Ltd
Methodological issues in nursing research Development of an evidence-based practice questionnaire for nurses

ledge and implementation of EBP (Upton & Lewis 1998) and 39 years age group (43%). A complete range of specialities was
has been used in studies with nurses (Upton 1999), medical included but the largest groups were surgery (12Æ5%), mental
staff, and other healthcare professionals (Curtin & Jaramazo- health (12Æ5%), and elder care (11Æ0%).
vic 2001). However, the psychometric properties of these
measures are rarely mentioned. It was to fulfil the need to
Data collection
develop a measure of attitudes towards, knowledge of and
implementation of EBP that the Evidence-Based Practice During the preliminary stage of questionnaire development
Questionnaire (EBPQ) described in this paper was developed. an item pool of statements and questions was produced
following a review of the literature and discussions with
healthcare professionals. This initial item pool was piloted
The study
and discussed with a group of 33 senior healthcare profes-
sionals, then reduced via item analysis and scaling methods.
Aim
Items were chosen from the item pool by calculating item-
The aim of the study was to develop and validate a self-report total correlation coefficients and retaining those with corre-
measure of the variables considered important for the lations of 0Æ4 or above. This method of item analysis assesses
implementation of EBP. the internal consistency of the scaling and is the one
recommended by Likert (1967).
This allowed a draft questionnaire to be developed that
Design
was, in turn, piloted and revised on two further occasions
The method of questionnaire development was based on that both by the initial group of healthcare professionals and by
proposed by Streiner and Norman (1995). An extensive item a steering group of experts in the field of health and social
pool was developed based on a literature search of key factors care policy. The healthcare professionals and steering group
influencing EBP, along with discussions with key health and established content validity of the questionnaire. The initial
social care professionals. This pool was refined by experienced questionnaire was sent to a stratified (by grade) sample of
healthcare professionals and face validity established. Principal 500 nurses. Data from this first survey were used to
component factor analysis was used to uncover the underlying establish the structure of the questionnaire. This new
dimensions of the scale. Internal consistency of the scale was measure was then sent to 500 different nurses for further
assessed by Cronbach’s a. Finally, construct validity was refinement and validation. Demographic details were sought
assessed via convergent and discriminant validity. from all respondents. In addition, respondents from the
second survey were asked to provide information on related
issues (e.g. knowledge of certain aspects of a local initiative
Participants
to promote EBP) for the purposes of validation. Comments
The first sample consisted of 500 nurses, who were randomly were also sought on the structure and content of the
selected from across Wales and sent a questionnaire requesting questionnaire.
their assistance. A second survey, with another sample of 500
nurses, was also conducted. In order to recruit a sample
Ethical considerations
representative of the whole of Wales, multi-stage cluster
sampling was used. In the first stage, hospitals and community The study was approved by a university ethics committee.
units were randomly selected from across Wales. The head of Letters explaining the study were sent with all questionnaires,
individual nursing units within each unit was then contacted and confidentiality was assured. Return of the completed
and, with their agreement, sufficient questionnaires for the questionnaires was taken as consent to participate.
number of qualified staff within the individual unit were
distributed. This procedure was repeated until the total number
Data analysis
of participants was obtained. From the two surveys a total of
751 nurses responded (an overall response rate of 75Æ1%). The All items were scored on a scale of 1–7, with a higher score
majority of respondents (36%) were employed at G grade indicating a more positive attitude towards clinical effective-
(Sister/Charge Nurse), although they ranged from C-grade ness/EBP, or use and knowledge of clinical effectiveness and
(Enrolled Nurses and nursing auxiliaries) through to I-grade EBP.
(Nurse Specialists). The majority were female, with only 9Æ8% Principal component analysis with an oblique rotation
male. The largest age group membership were in the 30– method was used to determine the structure of the question-

2006 The Authors. Journal compilation
2006 Blackwell Publishing Ltd 455
D. Upton and P. Upton

naire. Questions were considered as contributing to a subscale subscale, and for the attitude towards EBP subscale Cron-
if they had a factor loading of at least 0Æ4 on that factor. bach’s a was 0Æ79. Cronbach’s a was 0Æ91 for the knowledge/
Internal consistency was assessed using item-total correla- skills associated with EBP subscale.
tions and Cronbach’s a. Construct validity was assessed by
comparison of questionnaire scores and an independent
Validity
measure of awareness of a local clinical effectiveness initiat-
ive, using Pearson’s product–moment correlation. Discrimi- Construct validity
nant validity was assessed by comparison of subscale scores Construct validity was established by exploring the correla-
between respondents with knowledge of the local initiative tion between questionnaire scores and an independent
and those without this knowledge, using an independent measure of awareness of EBP, which involved asking
samples t-test. Statistical analysis was performed using SPSS respondents if they had any knowledge of certain aspects of a
11Æ0 (SPSS Inc., Chicago, IL, USA). local initiative to promote EBP. Correlation coefficients were
found to be in the range of 0Æ3–0Æ4 (P < 0Æ001), suggesting a
positive but moderate relationship.
Results
Discriminant validity
Survey 1
Discriminant validity was assessed by comparing those with
Questionnaire structure knowledge of the local initiative and those without. The re-
Data were subjected to principal component analysis, in sults indicated that those with knowledge of the local ini-
order to determine the factor structure of the questionnaire. tiative had a better attitude (t ¼ 2Æ5; d.f. ¼ 332; P < 0Æ01),
A Kaiser–Meyer–Okin score of 0Æ861 was achieved more frequent practice that was evidence-based (t ¼ 3Æ2;
(P < 0Æ001), suggesting that factor analysis was appropriate d.f. ¼ 360; P < 0Æ02) and (unsurprisingly) better knowledge
for this data set. Principal component factor analysis yielded of EBP (t ¼ 5Æ2; d.f. ¼ 360; P < 0Æ001).
three factors, each with an eigenvalue >1. The percentage of
explained variance for the first factor was 33Æ08% (‘practice
Feedback on the questionnaire
of EBP’: eigenvalue 3Æ97), for the second 17Æ07% (‘attitude
towards EBP’: eigenvalue 2Æ05) and for the third 11Æ63% Although the majority of respondents made no comment on
(‘knowledge/skills associated with EBP’: eigenvalue 1Æ40). the questionnaire itself, the feedback that was received was
The total amount of variance explained was 61Æ77% and generally encouraging. No one stated that the questionnaire
each of the items of the subscales loaded onto separate factors was too long or difficult to complete, although one person did
(see Table 1). Pearson’s correlations between each of the comment that asking about EBP was a ‘waste of time’. Many
subscales and total scores were all statistically significant. For respondents were pleased, however, to be asked these
the ‘practice of EBP’ subscale r ¼ 0Æ71 (P < 0Æ001), for the questions (e.g. ‘I welcome any means to assess my clinical
‘attitude towards clinical effectiveness/EBP’ r ¼ 0Æ95 effectiveness’ and ‘in order to be effective clinically it is
(P < 0Æ001), and for the ‘knowledge/skills associated with important to be self-critical’) and found the questionnaire
EBP’ subscale r ¼ 0Æ54 (P < 0Æ001). Cronbach’s a exceeded easy to complete (‘this questionnaire is clear and informative’
the accepted criterion of 0Æ70 for all subscales. and ‘this is a questionnaire of convenient and comfortable
format’). The good response rate also suggested that the final
questionnaire was received positively.
Survey 2

Questionnaire structure
Discussion
Factor analysis confirmed the three-scale structure established
by the analysis from the initial survey. The questionnaire described here is a self-report scale which
explores nurses’ day-to-day use of EBP. The EBPQ has been
shown to be user-friendly, quick and easy to complete, and to
Reliability
offer potential as a valid and reliable tool for measuring the
Internal consistency implementation of EBP.
Cronbach’s a was 0Æ87 for the entire questionnaire. Good The EBPQ has 24 items which are organized into three
internal reliability was also confirmed for each of the sub- subscales. Good internal reliability was found, with as
scales with Cronbach’s a being 0Æ85 for practice of EBP exceeding the 0Æ70 level for all three subscales as well as

456
2006 The Authors. Journal compilation
2006 Blackwell Publishing Ltd
Methodological issues in nursing research Development of an evidence-based practice questionnaire for nurses

Table 1 Results of factor analysis of the Evidence-Based Practice Questionnaire

Attitude Knowledge/skills
Practice of towards associated with
evidence-based evidence-based evidence-based
Item practice practice practice

How often have you formulated a clearly answerable question 0Æ490 0Æ280 0Æ325
as the beginning of the process towards filling this gap?
How often have you tracked down the relevant evidence once 0Æ691 0Æ324 0Æ428
you have formulated the question?
How often have you critically appraised, against set criteria, 0Æ578 0Æ301 0Æ079
any literature you have discovered?
How often have you integrated the evidence you have found 0Æ669 0Æ318 0Æ278
with your expertise?
How often have you evaluated the outcomes of your practice? 0Æ539 0Æ240 0Æ475
How often have you shared this information with colleagues? 0Æ601 0Æ162 0Æ497
My workload is too great for me to keep up-to-date with all the 0Æ232 0Æ398 0Æ057
new evidence
I resent having my clinical practice questioned 0Æ149 0Æ445 0Æ087
Evidence-based practice is a waste of time 0Æ297 0Æ550 0Æ015
I stick to tried and trusted methods rather than changing to 0Æ386 0Æ520 0Æ021
anything new
Research skills 0Æ297 0Æ037 0Æ636
IT skills 0Æ376 0Æ016 0Æ496
Monitoring and reviewing of practice skills 0Æ110 0Æ227 0Æ702
Converting your information needs into a research question 0Æ369 0Æ016 0Æ626
Awareness of major information types and sources 0Æ249 0Æ080 0Æ596
Ability to identify gaps in your professional practice 0Æ099 0Æ132 0Æ675
Knowledge of how to retrieve evidence 0Æ001 0Æ270 0Æ486
Ability to analyse critically evidence against set standards 0Æ229 0Æ140 0Æ736
Ability to determine how valid (close to the truth) the material is 0Æ052 0Æ297 0Æ516
Ability to determine how useful (clinically applicable) the 0Æ154 0Æ212 0Æ758
material is
Ability to apply information to individual cases 0Æ073 0Æ326 0Æ728
Sharing of ideas and information with colleagues 0Æ251 0Æ425 0Æ641
Dissemination of new ideas about care to colleagues 0Æ147 0Æ445 0Æ585
Ability to review your own practice 0Æ042 0Æ348 0Æ420

A bold figure indicates the highest loading and therefore the factor to which the item belongs.

the total score. Construct validity was established through the assessing healthcare workers’ use of EBP through a self-report
correlations shown with knowledge of a local initiative. The measure. The first of these is the frequency with which
moderate correlation between knowledge of the initiative and Sackett et al.’s (1997) steps (i.e. items 1–5 in Table 1) are
questionnaire scores suggested that the questionnaire was applied to individual patient care. Second, there is the
measuring attitudes related to EBP. Very high correlations are individual’s attitude towards EBP, which includes perceived
not desirable, as this would suggest that the questionnaire barriers such as workload, as well as personal judgements as
was simply measuring knowledge of the initiative rather than to the value of EBP. Finally, there is the self-rating of skills
a related concept and therefore would be providing little relevant to the implementation of EBP, which includes
additional information. research and information technology skills and the ability
A statistically significant difference was found between to interpret the literature and apply it to particular cases.
those with awareness of the local initiative and those without, The relationship between these different issues is a logical
indicating that the questionnaire has a good degree of one. If a person considers that they have none of the skills to
discriminant validity. implement EBP, then they are hardly likely to make use of it.
Examination of the items retained in the final questionnaire Equally, if they feel that there are too many other pressures
suggested that three different issues are important when on them (such as workload) or if they are not convinced by

2006 The Authors. Journal compilation
2006 Blackwell Publishing Ltd 457
D. Upton and P. Upton

ment initiatives for nurses and other healthcare professionals.

What is already known about this topic Furthermore, it may be of use in identifying individuals who
• Clinical effectiveness, achieved through evidence-based would most benefit from educational initiatives, although
practice, is essential to modern healthcare worldwide. further work on its psychometric properties is required in
• Barriers, including workforce attitudes and lack of time order to refine the measure.
and appropriate skills, may prevent evidence-based
practice.
Author contributions
• Currently no psychometrically validated tools exist to
measure the implementation of evidence-based practice. DU contributed to the study conception and design and also
obtained funding for this study. DU and PU contributed to
data collection and statistical expertise. PU contributed to
What this paper adds data analysis, drafting of the manuscript and provided
• Presents a valid and reliable self-report measure, which administrative, technical and material support.
considers three aspects of evidence-based practice: day-
to-day application, individual attitudes and relevant
References
skills.
• The sensitivity of the measure to change in attitudes Curtin M. & Jaramazovic E. (2001) Occupational therapists’ views
following educational initiatives needs to be estab- and perceptions of evidence based practice. British Journal of
Occupational Therapy 64, 214–222.
lished.
Davies H., Nutley S. & Mannion R. (2000) Organisational culture
and quality of health care. Quality in Healthcare 9, 111–119.
DoH (2000) The NHS Plan. The Stationery Office, London.
the merits of EBP, then it is again unlikely that they will Hughes J., Humphrey C., Rogers S. & Greenhalgh T. (2002) Evi-
follow Sackett’s steps when caring for individual patients. dence into Action: Changing Practice in Primary Care. Occasional
Indeed, these issues have been suggested by others to be Paper 84. Royal College of General Practitioners, London.
Likert R. (1967) The method of constructing an attitude scale. In
influencing the implementation of EBP (Metcalfe et al. 2001).
Readings in Attitude Theory and Measurement (Fishbein M., ed.),
What should perhaps be emphasized is that it is unimpor- Wiley, New York, pp. 90–95.
tant whether or not these individuals really do have these Metcalfe C., Lewin R., Wisher S., Perry S., Bannigan K., Klaber J. &
skills, or are overburdened by pressures of work, or if EBP is Moffet J. (2001) Barriers to implementing the evidence base in four
a valuable part of patient care – it is their perception that will NHS therapies: dieticians, occupational therapists, physiothera-
influence their behaviour in terms of EBP. As Davies et al. pists, speech and language therapists. Physiotherapy 87, 433–441.
O’Donnell C.A. (2004) Attitudes and knowledge of primary care
(2000) have shown, attitudes are part of the organizational
professionals towards evidence based practice: a postal survey.
culture and play a crucial role in determining the success or Journal of Evaluation in Clinical Practice 10, 197–205.
failure of workplace innovations. Ramos K., Schafer S. & Tracz S. (2003) Validation of the Fresno test
A self-report questionnaire for measuring the implementa- of competence in evidence based medicine. British Medical Journal
tion of EBP at grass roots levels is essential because, as 326, 319–321.
Sackett D.L., Richardson W.S., Rosenberg W. & Haynes R.B. (1997)
Hughes et al. (2002) have demonstrated, if EBP is to have any
Evidence Based Medicine. How to Practice and Teach EBM.
impact at all it must be implemented through a bottom-up Churchill Livingstone, London.
strategy which takes full account of level of skills, local Stevenson K., Lewis M. & Hay E. (2004) Do physiotherapists’ atti-
organization and so forth. tudes towards evidence based practice change as a result of an
Finally, whilst the construct validity and internal reliability evidence based programme? Journal of Evaluation in Clinical
of the EBPQ have been demonstrated in this study, we Practice 10, 207–217.
Streiner D. & Norman G. (eds) (1995) Health Measurement Scales.
recommend that further work is carried out to establish the
A Practical Guide to their Development and Use, 2nd edn. Oxford
sensitivity of the measure to change in attitudes following University Press, Oxford.
educational initiatives. Upton D. (1999) Attitudes towards, and knowledge of, clinical ef-
fectiveness in nurses, midwives, practice nurses and health visitors.
Journal of Advanced Nursing 29, 885–893.
Conclusion Upton D. & Lewis B.K. (1998) Evidence based practice: a ques-
tionnaire to assess knowledge, attitudes and practice. British
The EBPQ may be useful for developing and evaluating Journal of Therapy and Rehabilitation 5, 647–650.
educational programmes, policy developments and manage-

458
2006 The Authors. Journal compilation
2006 Blackwell Publishing Ltd