See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/5992662

e-Health Readiness Assessment Tools for Healthcare Institutions in Developing

Countries

Article  in  Telemedicine and e-Health · September 2007

DOI: 10.1089/tmj.2006.0064 · Source: PubMed

CITATIONS READS
148 5,552

6 authors, including:

Shariq Khoja Richard Scott

Tech4Life Enterprises University of KwaZulu-Natal
42 PUBLICATIONS   990 CITATIONS    151 PUBLICATIONS   2,892 CITATIONS   

SEE PROFILE SEE PROFILE

Ann Casebeer Mehnoor Mohsin

The University of Calgary Queen Mary, University of London
55 PUBLICATIONS   2,214 CITATIONS    28 PUBLICATIONS   363 CITATIONS   

SEE PROFILE SEE PROFILE

Some of the authors of this publication are also working on these related projects:

Community Health Information Tracking System View project

Smart Grid View project

All content following this page was uploaded by Richard Scott on 27 May 2014.

The user has requested enhancement of the downloaded file.

TELEMEDICINE AND e-HEALTH
Volume 13, Number 4, 2007
© Mary Ann Liebert, Inc.
DOI: 10.1089/tmj.2006.0064

Original Research

e-Health Readiness Assessment Tools for Healthcare

Institutions in Developing Countries

SHARIQ KHOJA, M.D., Ph.D.,1,2 RICHARD E. SCOTT, Ph.D.,1 ANN L. CASEBEER, Ph.D.,1
M. MOHSIN, M.Sc.,3 A.F.M. ISHAQ, Ph.D.,3 and SALMAN GILANI, M.B.B.S.4

ABSTRACT

e-Health Readiness refers to the preparedness of healthcare institutions or communities for

the anticipated change brought by programs related to Information and Communications
Technology (ICT). This paper presents e-Health Readiness assessment tools developed for
healthcare institutions in developing countries. The objectives of the overall study were to
develop e-health readiness assessment tools for public and private healthcare institutions in
developing countries, and to test these tools in Pakistan. Tools were developed using partic-
ipatory action research to capture partners’ opinions, reviewing existing tools, and develop-
ing a conceptual framework based on available literature on the determinants of access to e-
health. Separate tools were developed for managers and for healthcare providers to assess
e-health readiness within their institutions. The tools for managers and healthcare providers
contained 54 and 50 items, respectively. Each tool contained four categories of readiness. The
items in each category were distributed into sections, which either represented a determinant
of access to e-health, or an important aspect of planning. The conceptual framework, and the
validity and reliability testing of these tools are presented in separate papers. e-Health readi-
ness assessment tools for healthcare providers and managers have been developed for health-
care institutions in developing countries.

INTRODUCTION in a report from CANARIE Inc. as “the degree

to which users, healthcare institutions, and the

U SE OF INFORMATION AND COMMUNICATION

TECHNOLOGY (ICT) in healthcare is widely
perceived as an important source of reducing
healthcare system itself, are prepared to par-
ticipate and succeed with e-health implemen-
tation.”2 Most available literature describes e-
discrimination based on lack of access to in- readiness as a response to the growing digital
formation, and as a means of timely response divide between developed and developing
to matters impacting one’s personal or com- countries.3–6 Studies also suggest that the main
munity health.1 The idea of e-readiness in intent of e-readiness assessment is to provide a
healthcare (hereto referred as “e-health readi- unified framework to evaluate dimensions of
ness”) is relatively new, and has been defined the digital divide.7 In contrast, very little liter-

1Facultyof Medicine, University of Calgary, Calgary, Alberta, Canada.

2Aga Khan University, Karachi, Pakistan.
3COMSATS Institute of Information Technology, Islamabad.
4Holy Family Hospital, Rawalpindi, Pakistan.

425
426 KHOJA ET AL.

ature refers to e-health readiness. Available re- MATERIALS AND METHODS

ports indicate e-health readiness is measured
by assessing the relative status of governments, This overall study used a “mixed methods”
healthcare institutions, or users in areas most approach, applying a sequential exploratory
critical for adoption and success of programs design to develop and validate e-health readi-
using ICT.2 This represents an important step ness assessment tools.14,15 The sequential ex-
in change management, and including this step ploratory design is best suited for developing
in the planning process increases the chances and testing new instruments, particularly
of program success as well as enhancing equity when researchers look for accuracy, precision,
and reducing the digital divide. e-Health readi- and application of the instrument at the same
ness assessment could also provide other ad- time. According to Tashakkori, the sequential
vantages, such as: (1) avoiding huge losses in exploratory design “is better suited to explore
time, money, and effort; (2) avoiding delays a phenomenon . . . and is often discussed as a
and disappointments among planners, staff, design used when a researcher develops and
and users of services; and (3) facilitating the tests an instrument.”14
process of change in the institutions and com- In the process of using a sequential ex-
munities involved, from the stage of precon- ploratory design, the study first applied par-
templation (firmness and resistance to change) ticipatory action research to gather expert opin-
through contemplation (acceptance of new ion on the composition and format of e-health
ideas) and to preparation (preparedness for readiness assessment tools. The study also used
change).4,8 Since most of these advantages of e- perspectives from existing e-readiness and e-
health readiness are yet to be proven, it is im- health readiness tools and the available litera-
portant to first improve the capacity of institu- ture. Two e-health readiness assessment tools
tions to assess their e-health readiness. Only were developed for application in healthcare
then would they be able to ascertain the role institutions of developing countries: one for
and impact of e-health readiness in the broader managers, and one for healthcare providers.
implementation of e-health. Several e-readi- Details of the conceptual framework used for
ness assessment tools have been developed in developing the tools will be published in a sep-
areas such as e-business, e-commerce, e-learn- arate article.
ing, and e-government.4,6,9 Most of these tools To validate the tools, the overall study first
claim to be useful both in diagnosing current used a qualitative method, borrowed from the
situations in terms of an organization’s or com- tradition of case-study, by employing in-depth
munity’s preparedness to implement ICT, and semistructured interviews to test face and con-
also in orienting steps to narrow the digital di- tent validity.16,17 Details of validity testing are
vide.7 published in a separate article.12 This testing
Efforts to develop readiness assessment tools demonstrated very good face and content va-
for e-health programs have taken place in the lidity for both tools. To assess their reliability,
Canadian provinces of Alberta and Ontario a survey methodology was used. All managers
where tools to assess e-health readiness in and healthcare providers working in the par-
healthcare institutions and communities were ticipating institutions, and having experience
developed.10,11 Despite being very comprehen- with the planning or implementation of e-
sive, these tools focus on developed countries, health programs, were asked to complete the
and do not address the determinants of re- e-health readiness assessment tools developed
duced ICT accessibility in developing coun- in this study. Data were analyzed to determine
tries. It is, therefore, important to create tools the internal consistency by calculating Cron-
that are more specific to the issues faced by de- bach’s alpha for each tool and also for individ-
veloping countries. This paper describes the ual categories within the tools.18 In the final
process of design and construction of e-health step, both qualitative and quantitative results
readiness assessment tools developed for were incorporated to extend the interpretation
healthcare institutions in developing countries. of results. Details of reliability testing are also
Details of validity and reliability testing in Pak- published separately.13 This testing demon-
istan are provided elsewhere.12,13 strated very high reliability for both tools.
ASSESSMENT TOOLS FOR HEALTHCARE INSTITUTIONS 427

Ethics approval for the study was obtained team of researchers from University of Calgary
from the “Conjoint Health Research Ethics and partner institutions in Pakistan. Based
Board” at the University of Calgary. upon the literature, existing tools, and expert
opinion, the research team agreed on both the
composition and the format of the draft e-
RESULTS health readiness assessment tools.
All the items addressing a single determinant
Final drafts of both tools, for managers and of access to e-health or a single aspect of plan-
healthcare providers, were approved by the ning were grouped in separate sections. There

TABLE 1. COMPOSITION AND FORMAT OF CATEGORY OF CORE-READINESS

Score

Statements: 1 2 3 4 5 D/K

Identification of Needs for future changes, which the proposed telehealth/

e-health project will address:
1. Organization has properly identified its needs 1 2 3 4 5 D/K
2. Organization has properly prioritized its needs 1 2 3 4 5 D/K
Dissatisfaction with status quo on the prioritized needs (related to the
proposed project):
1. There is general dissatisfaction with current handling of issues that could be 1 2 3 4 5 D/K
addressed through telehealth/e-health
2. Solutions other than telehealth/e-health have been explored. 1 2 3 4 5 D/K
Awareness about telehealth/e-health in the organization:
1. Awareness of ICT and internet’s role in healthcare exists among the planners 1 2 3 4 5 D/K
2. Awareness of ICT and internet’s role in addressing the prioritized needs 1 2 3 4 5 D/K
exists among the planners.
Comfort with technology:
1. There is general comfort in using ICT/internet among users of the proposed 1 2 3 4 5 D/K
telehealth/e-health project.
2. There is general comfort among staff in using ICT/internet for storing 1 2 3 4 5 D/K
patient information.
3. There is general comfort among staff in using ICT/internet for the purpose 1 2 3 4 5 D/K
of patient care and education.
Trust on the use of ICT:
1. All the policymakers and senior administrators trust new technology as a 1 2 3 4 5 D/K
solution to the identified problems
2. All the staff members trust new technology as a solution to the identified 1 2 3 4 5 D/K
problems
3. There are plans in place to increase staff’s trust and confidence in the new 1 2 3 4 5 D/K
technology
Planning for the new telehealth/e-health project:
1. An individual or a group has taken responsibility for planning. 1 2 3 4 5 D/K
2. All the user groups among staff and other stakeholders have been involved 1 2 3 4 5 D/K
in planning
3. There is an appropriate plan for implementation of telehealth/e-health 1 2 3 4 5 D/K
initiative
4. The implementation plan includes proper budgeting and identification of 1 2 3 4 5 D/K
resources.
5. There is an appropriate plan for evaluation of telehealth/e-health initiative, 1 2 3 4 5 D/K
including option for external evaluation
Overall satisfaction and willingness:
1. The proposed technology is appropriate according to the conditions within 1 2 3 4 5 D/K
the organization
2. There is a willingness among staff to implement the technology for its 1 2 3 4 5 D/K
intended purpose
Integration of technology:
1. Integration of technology with the current services has been considered in 1 2 3 4 5 D/K
the planning process
2. There is a plan in place to integrate telehealth/e-health with the current 1 2 3 4 5 D/K
services
428 KHOJA ET AL.

were one to four items in each section. The sec- Category II—Technological readiness
tions were then grouped into four categories
This category was included in the tool for man-
for each tool; core-readiness, societal readiness,
agers only, and addressed the availability and af-
and policy readiness were common to both the
fordability of required ICT, and the hardware
tools. The fourth category in the tool for man-
and software needed to implement a proposed
agers was called “technological readiness,”
program. Specific technological readiness items
whereas the fourth category in the tool for
dealt with physical access to technology along
healthcare providers was called “learning
with determinants of accessibility such as af-
readiness.”
fordability and capacity building (Table 2).
Draft tools were tested for validity and reli-
ability.12,13 The final tools contained 54 items
Category III—Learning readiness
for managers and 50 items for healthcare
providers. Each of the categories, along with all This category was included in the tool for
the items, is described below and illustrated in healthcare providers only, and addressed is-
Tables 1–5. sues related to the existence of programs and
resources to provide training to healthcare
Category I—Core readiness providers in using the technology. Specific
learning readiness items dealt with the inclu-
This category was common in both the tools,
sion of healthcare providers in the planning
and addressed the overall planning process for
process and determinants of accessibility such
a proposed e-health program, and the knowl-
as capacity building (Table 3).
edge and experience of planners with pro-
grams using ICT. Specific core-readiness items
Category IV—Societal readiness (ICT use
dealt with the importance of needs assessment,
and interaction)
key aspects of planning, and the determinants
of accessibility such as appropriateness of tech- This category was included in both the tools,
nology, and integration of technology with ex- and dealt with any existing interaction of the
isting services (Table 1). concerned institution with other healthcare in-

TABLE 2. COMPOSITION AND FORMAT OF CATEGORY OF TECHNOLOGICAL READINESS

Score

Statements: 1 2 3 4 5 D/K

Speed and quality of ICT/Internet at the institution:

1. Speed of connections is appropriate for the proposed use 1 2 3 4 5 D/K
2. Quality of connections is appropriate for the proposed use 1 2 3 4 5 D/K
Service/Support for ICT:
1. Service/support is available within a reasonable time frame for the 1 2 3 4 5 D/K
proposed use.
2. Local support is proficient to address most of the problems related 1 2 3 4 5 D/K
to the proposed use.
Hardware and software:
1. Hardware and software required for the proposed project are 1 2 3 4 5 D/K
readily available.
2. Hardware and software required for the proposed project are 1 2 3 4 5 D/K
readily affordable.
Availability and affordability of the desired ICT
1. Required ICT (telephone/internet/bandwidth) is easily available 1 2 3 4 5 D/K
for the institution.
2. Required ICT (telephone/internet/bandwidth) is easily available 1 2 3 4 5 D/K
for the institutions involved.
Institutional access to ICT/Internet training:
1. Programs are in place to train the users for proposed project. 1 2 3 4 5 D/K
2. Manpower is in place to train the users for proposed project. 1 2 3 4 5 D/K
ASSESSMENT TOOLS FOR HEALTHCARE INSTITUTIONS 429

TABLE 3. COMPOSITION AND FORMAT OF CATEGORY OF LEARNING READINESS

Score

Statements: 1 2 3 4 5 D/K

ICT/Internet training for healthcare providers:

1. Personnel and programs are in place for training 1 2 3 4 5 D/K
Use of ICT/Internet to enhance education of care providers:
1. Programs exist for continuous education 1 2 3 4 5 D/K
2. ICT/Internet is readily used in continuous education 1 2 3 4 5 D/K
3. Programs are in place to use ICT/Internet for continuous education 1 2 3 4 5 D/K
Involvement of healthcare providers in telehealth/e-health projects:
1. There is a plan in place to involve healthcare providers in the planning 1 2 3 4 5 D/K
of new telehealth/e-health interventions.
2. There is a plan in place to involve healthcare providers in the 1 2 3 4 5 D/K
implementation of new telehealth/e-health interventions.

stitutions in the region and beyond. Specific so- at the government and institutional levels to
cietal readiness items dealt with determinants of address common issues such as licensing, lia-
accessibility such as relevance of content and so- bility, and reimbursement. Specific policy
ciocultural factors, and addressed the issues of readiness items dealt with determinants of ac-
inequity in gender and social classes (Table 4). cessibility such as the legal and regulatory
framework and political will (Table 5).
Category V—Policy readiness (at institutional
and government levels) Scoring of e-health readiness assessment tools
This category was also included in both the Each of the categories in both the tools con-
tools, and dealt with the existence of policies tained items that determine readiness of an in-

TABLE 4. COMPOSITION AND FORMAT OF CATEGORY OF SOCIETAL READINESS

Score

Statements: 1 2 3 4 5 D/K

Communication with other organizations:

1. Staff regularly uses ICT/Internet to communicate with staff at the other health 1 2 3 4 5 D/K
institutions of the region
2. Staff regularly uses ICT/Internet to communicate with local community and clients 1 2 3 4 5 D/K
3. Other institutions involved in the telehealth/e-health project have also planned to 1 2 3 4 5 D/K
go through e-readiness assessment.
[For e-learning projects] Sharing of locally relevant content between healthcare
institutions:
1. Material on locally relevant health issues is shared between this institution and 1 2 3 4 5 D/K
other institutions.
2. The relevant material is available in language(s) easily understood by all the 1 2 3 4 5 D/K
concerned staff and other users of information
[For service related projects] Providing care to patients and communities in
collaboration with other healthcare institutions:
1. A referral system is available between this institution and other healthcare 1 2 3 4 5 D/K
institutions to provide patient care in certain specialities.
2. ICT is currently used for referrals between this institution and other healthcare 1 2 3 4 5 D/K
institutions.
Considering sociocultural factors among staff:
1. Both the genders have equal and unrestricted access to the technology. 1 2 3 4 5 D/K
2. Staff from all levels get direct benefit from the use of technology. 1 2 3 4 5 D/K
Considering sociocultural factors among clients and communities:
1. Use of ICT will benefit men and women equally in the society. 1 2 3 4 5 D/K
2. People from all socioeconomic strata get direct benefit from the use of technology. 1 2 3 4 5 D/K
430 KHOJA ET AL.

TABLE 5. COMPOSITION AND FORMAT OF CATEGORY OF POLICY READINESS

Score

Statements: 1 2 3 4 5 D/K

ICT related regulations:

1. Government policies are in place to promote and manage use of telehealth/e-health 1 2 3 4 5 D/K
in healthcare institutions.
2. Institutional policies are in place to promote and manage use of telehealth/e-health 1 2 3 4 5 D/K
in your institution.
Policies regarding licensure and liability:
1. Government policies are in place to allow care provision in other jurisdictions 1 2 3 4 5 D/K
through telehealth.
2. Institutional policies are in place to allow care provision in other jurisdictions 1 2 3 4 5 D/K
through telehealth.
3. Government policies are in place to deal with liability issues. 1 2 3 4 5 D/K
4. Institutional policies are in place to deal with liability issues. 1 2 3 4 5 D/K
Policies regarding reimbursement:
1. Government policies are in place to ensure proper reimbursement to the 1 2 3 4 5 D/K
healthcare providers in your institution.
2. Institutional policies are in place to ensure proper reimbursement to the 1 2 3 4 5 D/K
healthcare providers in your institution.
Awareness and support of ICT among politicians:
1. Politicians are generally aware of the benefits of ICT use in healthcare. 1 2 3 4 5 D/K
2. Politicians generally support the use of ICT use in healthcare. 1 2 3 4 5 D/K
Awareness and support of ICT among policymakers at the institutional level:
1. Policy makers are aware of the benefits of ICT in healthcare institutions. 1 2 3 4 5 D/K
2. Policy makers support the use of ICT in healthcare institutions. 1 2 3 4 5 D/K

stitution from the perspectives of managers or The e-health readiness assessment tools de-
healthcare providers. Each item could be rated veloped and validated within our study are in-
by the respondent using a five-point Likert- tended for managers and healthcare providers
type scale. The total score was intended to pro- to use independently for the planning of e-
vide longitudinal assessment that would place health programs in their own healthcare insti-
an institution at different stages between pre- tutions. These tools deal with factors that need
pared and not prepared, and was not intended to be addressed in the planning of e-health pro-
for interinstitutional comparison. grams, and therefore help planners evaluate
their status and progress at various stages,
without comparison to other healthcare insti-
DISCUSSION tutions. Validity and reliability testing of these
tools has taken place with different levels of in-
In order to comprehensively assess e-health stitutions, sectors, and geographical regions,
readiness of healthcare institutions in develop- making these tools more reliable and valid for
ing countries, the tools must address the key de- use in healthcare institutions throughout Pak-
terminants of accessibility to e-health relevant to istan. The tools have also been translated into
their context, along with other key planning is- the country’s local language (Urdu) to increase
sues such as needs assessment, implementation, their usability in more peripheral areas of the
and evaluation. Consideration of these issues country, where language can otherwise be a
should allow greater access to e-health for staff significant barrier to participation and pro-
of different genders and levels of service, and gress.
clients of different genders and socioeconomic The availability of e-health readiness as-
strata. Improving these issues of inequity in ac- sessment tools should help in improving the
cess to e-health could, in turn, help in address- quality of planning of e-health programs in
ing the issue of the digital divide, which is so healthcare institutions of Pakistan, as well as
prominent in the developing country setting. increasing the awareness, confidence, and trust
ASSESSMENT TOOLS FOR HEALTHCARE INSTITUTIONS 431

among staff and planners associated with e- 6. See www.cspp.org. (Last accessed April 30, 2006).
health applications already in use. The tools 7. Arce ME, Hopmann C. The concept of eReadiness
and its application in developing countries: Method-
should also help address technology and learn-
ological problems and results for the eReadiness
ing issues among staff, support improved com- in Nicaragua. 2002. Managua-Nicaragua. 2002.
munication between the healthcare institu- Available at http://www.nsrc.org/CENTRAM/NI/
tions, the clients, and the providers of care, and eReadiness-developing-countries.pdf (last accessed
also reduce barriers to the use of ICT related to December 15, 2005).
gender and socioeconomic conditions. With ap- 8. Prochaska JM, Prochaska JO, Levesque DA. A trans-
theoretical approach to changing organizations. Ad-
propriate contextual assessment, it is antici- ministrat Policy Mental Health 2001;28:247–261.
pated the e-health readiness assessment tools 9. See www.bridges.org. (Last accessed April 30, 2006).
presented here will find broader applicability 10. Jennett P, Jackson A, Healy T, Ho K, Kazanjian A,
in other developing countries, better preparing Woolland R, Hayot S, Bates J. A study of rural com-
them for introduction of e-health solutions. munity’s readiness for telehealth. J Telemed Telecare
2003;9:259–263.
11. Flewelling C, Ingram CA. Telepediatrics in Canada:
An overview. Telemed J E-Health 2004;10:357–368.
ACKNOWLEDGMENTS 12. Khoja S, Casebeer AL, Scott RE, Gilani SN. Validat-
ing e-Health readiness assessment tools for develop-
Funding for this article was by the Canadian ing countries. Telemed e-Health 2006;3(1):24–30.
Institute for Health Research and International 13. Khoja S, Scott RE, Mohsin M, Ishaq AFM. Testing re-
Development Research Center. Partners in the liability of e-Health readiness assessment tools for
developing countries. e-Health Int 2006;3(1):31–37.
study were the COMSATS Institute of Infor-
14. Tashakkori A. Handbook of mixed methods in social and
mation Technology, Islamabad; Telemedicine behavioral research. 1st ed. Thousand Oaks, CA: Sage
Association of Pakistan; and ProtoMed Inc., Publications, 2003:169–293.
Pakistan. 15. Sandelowski M. Combining qualitative and quantita-
tive sampling, data collection, and analysis tech-
niques in mixed-method studies. Res Nurs Health
REFERENCES 2000;23:246–255.
16. Creswell JW. Qualitative inquiry and research design:
Choosing among five traditions. 1st ed. Thousand Oaks,
1. Tsang MW, Mok M, Kam G, Jung M, Tang A, Chan
CA: Sage Publications 2003;13–26.
U, Chu CM, Li I, Chan J. Improvement in diabetes
17. Yin RK. Enhancing the quality of case studies in
control with a monitoring system based on a hand-
health services research. Health Serv Res 1999;(5 Pt
held, touch-screen electronic diary. J Telemed Telecare
2):1209–1224.
2001;7:47–50.
18. Streiner DL. A checklist for evaluating the usefulness
2. See http://www.fp.ucalgary.ca/telehealth/Canarie%
of rating scales. Can J Psychiatry 1993;38:140–148.
20-%20Final%20Report,%20June%202002/Canarie
%20-%20Final%20Report,%20June%202002.pdf (Last
accessed April 30, 2006).
3. Masi CM, Suarez-Balcazar Y, Cassey MZ, Kinney L,
Address reprint requests to:
Piotrowski ZH. Internet access and empowerment: A Shariq Khoja, M.D., Ph.D.
community-based health initiative. J Gen Intern Med Department of Community Health Sciences
2003;18:525–530. P.O. Box 3500
4. See www.readinessguide.org (Last accessed April 30, Stadium Road
2006).
The Aga Khan University
5. Kalichman SC, Weinhardt L, Benotsch E, Cherry C.
Closing the digital divide in HIV/AIDS care: Devel- Karachi, 74800, Pakistan
opment of a theory-based intervention to increase In-
ternet access. AIDS Care 2002;14:523–537. E-mail: shariq.khoja@aku.edu
View publication stats