See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/265140668

Evaluating E-Government

Chapter · January 2014

DOI: 10.4018/978-1-4666-5129-6.ch001

CITATIONS READS
3 6,144

1 author:

Fotini Patsioura
University of Macedonia
18 PUBLICATIONS 127 CITATIONS

SEE PROFILE

All content following this page was uploaded by Fotini Patsioura on 17 May 2019.

The user has requested enhancement of the downloaded file.

Evaluating Websites and
Web Services:
Interdisciplinary Perspectives on
User Satisfaction
Denis Yannacopoulos
Technological Educational Institute of Piraeus, Greece

Panagiotis Manolitzas
Technical University of Crete, Greece

Nikolaos Matsatsinis
Technical University of Crete, Greece

Evangelos Grigoroudis
Technical University of Crete, Greece

A volume in the Advances in Web

Technologies and Engineering (AWTE)
Book Series
Managing Director: Lindsay Johnston
Production Manager: Jennifer Yoder
Development Editor: Austin DeMarco
Acquisitions Editor: Kayla Wolfe
Typesetter: Christina Barkanic
Cover Design: Jason Mull

Published in the United States of America by

Information Science Reference (an imprint of IGI Global)
701 E. Chocolate Avenue
Hershey PA 17033
Tel: 717-533-8845
Fax: 717-533-8661
E-mail: cust@igi-global.com
Web site: http://www.igi-global.com

Copyright © 2014 by IGI Global. All rights reserved. No part of this publication may be reproduced, stored or distributed in
any form or by any means, electronic or mechanical, including photocopying, without written permission from the publisher.
Product or company names used in this set are for identification purposes only. Inclusion of the names of the products or
companies does not indicate a claim of ownership by IGI Global of the trademark or registered trademark.

Library of Congress Cataloging-in-Publication Data

Evaluating websites and web services : interdisciplinary perspectives on user satisfaction / Denis Yannacopoulos, Panagiotis
Manolitzas, Nikolaos Matsatsinis, and Evangelos Grigoroudis, editors.
pages cm
Includes bibliographical references and index.
ISBN 978-1-4666-5129-6 (hardcover) -- ISBN 978-1-4666-5130-2 (ebook) -- ISBN 978-1-4666-5132-6 (print & perpetual
access) 1. Web sites--Design--Evaluation. 2. Web services--Evaluation. 3. Internet users. I. Yannacopoulos, Denis, 1949-
TK5105.888.E926 2014
006.7--dc23
2013046354

This book is published in the IGI Global book series Advances in Web Technologies and Engineering (AWTE) (ISSN:
Pending; eISSN: pending)

British Cataloguing in Publication Data

A Cataloguing in Publication record for this book is available from the British Library.

All work contributed to this book is new, previously-unpublished material. The views expressed in this book are those of the
authors, but not necessarily of the publisher.

For electronic access to this publication, please contact: eresources@igi-global.com.

 1

Chapter 1
Evaluating E-Government
Fotini Patsioura
University of Macedonia, Greece

ABSTRACT
Over the past few years, e-government growth has resulted in the development of many state and local
e-government initiatives in both developing and industrialised countries. In most cases, e-government
programs incorporate the implementation and operation of Web portals and Websites, putting information
online, transforming processes to Web-based transactions, interacting with citizens and companies. While
e-government Websites are evolving, evaluation and effectiveness assessments are emerging processes in
order to optimise digital services and enhance the engagement of citizens and companies to government
administrative processes for their own benefit. So far, many different measures of governmental Website
effectiveness have been developed to address strategic, marketing, and design issues. This chapter aims to
provide an insight on the theory and application of evaluation and effectiveness approaches with regards
to Web portals and Websites implemented to operate under central or local government authorities.
Both quantitative and qualitative aspects are analysed, addressing operational and functional tactics.

INTRODUCTION transactions with private individuals, businesses

and other stakeholders in the e-government en-
E-Government is the use of Information and Com- vironment.
munication Technology (ICT) and particularly So far, e-government web sites remain the
Internet-based and web-based telecommunica- main critical information resource and service
tion practices to facilitate connections within delivery medium. Because of the diversity and
and between the authorities and agencies of the range of e-government web sites, a typology of
Public Sector and also deliver government critical them is required to address issues with regards to
information and digital services to both citizens business models’ analysis, strategic design, man-
and companies. Various e-Government business agement and evolution. With most e-government
models have been implemented and matured to en- projects in the maturity phase of their life cycle,
hance the efficiency of public sector organisational effectiveness assessment is becoming vital for the
processes by supporting transactions between optimisation of their performance. Particularly in
the employees in government agencies and also the e-government area, evaluation and effective-

DOI: 10.4018/978-1-4666-5129-6.ch001

Copyright © 2014, IGI Global. Copying or distributing in print or electronic forms without written permission of IGI Global is prohibited.
 Evaluating E-Government

ness measures of public sector web portals and DEFINING E-GOVERNMENT

sites are both quantitative and qualitative. While
quantitative measures examine the level of adop- What is E-Government
tion and impact of the web sites in question with
the use of e-metrics and user-satisfaction indices, The e-government idea was introduced by a former
qualitative measures have a different dual purpose: U.S vice president and includes two concepts:
firstly, to investigate and assess the government firstly the governments’ obligation to provide a
web sites performance in terms of functionality link to citizens for getting services from various
and usability and secondly to identify informa- government agencies in an automatic way and
tion and system qualitative factors in evaluating secondly the governments’ need to reduce cost
e-services delivered by the web sites in question. and improve performance with the use of infor-
The European Commission of the European mation and communication networks among its
Union and the U.S.A E-Government Resource authorities (Almarabeh & AbuAli, 2010). Despite
Centre have provided guidelines and standards for its short history (e-government emerged in the late
effective federal, state and local government web 90’s), the e-government concept has been well
sites addressing analysis, design and evaluation defined by academics and researchers. Different
issues. However, in many cases governmental definitions of e-government focus on different
web portals and sites have failed to meet their elements and dimensions of government or gover-
objectives. In this context, the implementation of nance emphasizing on its strategic role. However,
evaluation frameworks is vital as it will contribute in most definitions, the use of Information and
to the identification of the key performance factors Communication Technology (ICT) is described
and indicators for better and further e-government as the key enabler for all e-government projects.
development. It is clear, that each definition describes the
It is a common belief among academics and environment formed in that particular period of
practitioners that e-government is now growing in time by the advances and applications of ICT and
a fast manner. Government web sites are transform- the e-government initiatives in progress. Table 1
ing, from basic online presence providing mainly includes definitions to capture the evolution of
information and a medium level of interactivity e-government over the past ten years.
to fully transactional channels. Meanwhile, Web Bannister’s (2007) definition of e-government
2.0 government has strengthened the relationship as the “delivery of government services over the
between governments and the citizens or compa- internet in general and the Web in particular”
nies. For public and government administration describes best the e-government evolution with
authorities that have already stepped in the digital most e-government projects implementing and
era proving information, communication and ser- supporting web-based applications through the
vices through the operation web portals and web development and exploitation of public portals
sites, web site evaluation should be a continuous and web sites. As ICT advances, e-government
process to achieve efficiency of their performance. portals and web sites are becoming more complex
in architecture and structure supporting multiple
business models.

2
Evaluating E-Government

Table 1. E-government definitions

Year Definition Author

2000 E-government is the continuous optimization of service delivery, constituency participation and Gartner Group (Baum
governance by transforming internal and external relationships through technology, the Internet and & Di Maio)
new media.
2001 A government’s use of technology, such as the Internet, to aid the delivery of information and services Layne & Lee
to citizens, employees, business partners, other agencies and other government entities.
2004 The use of ICT in public administrations combined with organisational change and new skills in order European Information
to improve public services, democratic processes and strengthen support to the public policies. Society
2004 E-Government refers to the use by government agencies of information technologies that have World Bank
the ability to transform relations with citizens, businesses, and other arms of government. These
technologies can serve a variety of different ends: better delivery of government services to citizens,
improved interactions with business and industry, citizen empowerment through access to information,
or more efficient government management.
2007 electronic Government’ (or in short ‘e-Government’) essentially refers to ‘The utilization of Jeong
Information Technology (IT), Information and Communication Technologies (ICTs), and other web-
based telecommunication technologies to improve and/or enhance on the efficiency and effectiveness
of service delivery in the public sector.
2011 E-government refers to the use of information and communication technologies (ICT) - such as Wide The United Nations
Area Networks, the Internet, and mobile computing - by government agencies.

E-Government Business and base to accomplish and optimise coordina-

Service Delivery Models tion and cooperation between local, de-
partmental and national authorities
The scientific field of e-government is almost 20 • G2E (Government to Employee) is the em-
years old with official governmental web sites ployment of basically information and oth-
first appearing in the mid 1990’s (Coursey & er back office systems to support the main
Norris, 2008). However, several business mod- stakeholder of internal governmental body,
els were formed and analysed in literature. An the employee
e-government business model identifies the key • G2B (Government to Business) describes
stakeholders in the e-government environment all online interactions and transactions be-
and their interrelationships and specifies the key tween the government agencies and private
business processes that represent the integration sector companies. Two of its main forms
of governmental information and services. are e-procurement and auctioning of gov-
Figure 1 presents the main business models ernment oversupplies
of e-government with regards to outside and • B2G (Business to Government) refers to
inside the government stakeholders based on the public sector businesses selling services
academic literature (Holmes, 2001; Fang, 2002; and products to government agencies with
Beyon - Davies, 2007; Palvia & Shalma, 2007; the use of Internet and ICTs
Albarabeh et al, 2010): • G2C (Government to Citizen) remains the
business model of most e-government ini-
• G2G (Government to Government) refers tiatives in which the government provides
to ICT applications (information systems, online access to information and digital
intranets, data & communication networks, services to its main outside stakeholders
Internet services) providing a common (the citizen)
communication and information exchange

3
 Evaluating E-Government

Figure1. E-government business models

• C2G (Citizen to Government) refers to the the e-government evolution require more advanced
two –way communication activities which web based applications to enable the citizens’ e-
allows citizens to interact online with their participation and therefore support consultative
government requesting answers and solu- and participatory activities (Reddick, 2011).
tions or providing feedback with regards to
the government agencies’ performance E-Government and E-Commerce:
• C2C (Citizen to Citizen) describes a future Similarities and Differences
perspective of e-government with citizens
interacting with each other forming an on- E-commerce and e-government are subsets of
line community to address governmental e-business adopting technological innovations to
issues transform their internal and external relations for
their own and their stakeholders benefit. E-com-
In recent times, web based software platforms merce proceeds e-government and this is why early
such as public portals, web sites and digital services e-government initiatives have adopted concepts,
remain a common place for the implementation of technologies and processes already applied online
e-government policies with outside stakeholders. in the private sector. Therefore, the comparison
The establishment of local, regional and national between e-government and e-commerce could be
web sites is the main outlet to integrate G2B and considered justified and constructive uncovering
G2C activities. Companies and citizens interact- the deficiencies and benefits of the two concepts
ing with government web sites have convenient correlation in theory and practice.
access to textual and multimedia information and Similar to e-commerce companies that use
digital services outside of office hours avoiding ICT to conduct business with other companies
long waits in queues to have a face to face meet- (B2B), engage customers (B2C) and support the
ing with civil servants (Fairweather & Rogerson, work of their employees (B2E), organisations with
2006). In addition, C2C interactions as a result of e-government activities enable the optimisation of

4
Evaluating E-Government

interagency coordination and cooperation (G2G), formed a comparative analysis between e-govern-
improve communication and transactions with ment and e-business web sites (the main objective
citizens (G2C) or organizations (G2B) and provide of these sites is the dissemination of information
public servants with the appropriate technical tools similar to e-government web sites). Outcomes of
to increase their productivity (G2E) (Fang, 2002; their research support the idea that e-government
Scholl et al, 2010) (Figure2). can learn from the private sector but should take
It is evident that e-government business mod- under consideration the differences of their target
els have been influenced by commercial business group’s needs and expectations.
models on the Internet as they share the same
social and technological environment (Carter & E-Government Web Sites Typology
Belanger, 2004). However, the drivers and pri-
orities of e-government affected by its political So far, in the area of e-government there is not
concept formed different to commercial online an accepted government web site classification or
relationships (Warkentin, 2002). typology. This is due to the diversity in scope and
Comparative analyses between e-commerce range of the implemented e-government projects.
and e-government outlets are limited in literature However, a categorisation of government portals
review. Table 2 summarises differences and simi- and web sites is necessary in order to understand
larities of the two concepts identified in previous their strategic and organisational objectives.
research (Jorgenson & Cable, 2002; AL-Shehry Two criteria are used to classify current e-
et al, 2006; Stahl, 2008; Scholl et al, 2009). government web sites (Figure 3):
On the other hand, many state that a com-
parison between e-commerce and e-government • Level of e-government: based on this crite-
web sites is irrelevant since payment transaction rion, government web sites could be local,
is the main interaction format of commercial state/provincial, regional, national or inter-
web-sites. In addition, the political nature of e- national (Albarabeh and Abuali, 2010).
government policies is also an important distin- • Level of growth (maturity)/service de-
guishing factor between commercial and govern- velopment: A lot of growth and maturity
ment web sites (Warkentin et al, 2002). In this models were implemented to conceptu-
context, Morgeson III and Mithas (2009) per- alise the steps or stages of the government

Figure 2. E-Commerce and e-government business models

5
 Evaluating E-Government

Table 2.Similarities and differences between e-commerce and e-government

Elements of Comparison E-Commerce E-Government Similar Different

Motivation Make profit Maximise social utility, create a
e-participation
Cost reduction of service delivery Cost reduction of service delivery a
Automation of internal processes Automation of internal processes a
Objectives Sale of products and services Optimisation of service quality to a
citizens
Information provision Information provision a
Online Customer service Online services to citizens a
Priority Safe & secure transactions Minimise digital divide a
Technology Internet, Web Based Platforms, Back Internet, Web based platforms, back a
Office Systems office systems
Decision Making Authority Centralised Dispersion of authority a
Target Group customers, potential customers Any Citizen a
Legislation Freedom laws and regulations restrictions and a
complexity
Services Primarily transactional Primarily informational a

Figure 3. E-government websites typology

web sites implementation. One of the first enable transactions, 3)vertical integration
classification of e-government develop- (local systems are linked to higher lever
ment, commonly referred to in many rela- ones) 4)horizontal integration (an inter-
tive studies was introduced by Layne and connection between systems across sever
Lee (2001), identifying four stages of de- government agencies is accomplished).
velopment, 1)cataloguing (information on In the same context, the United Nations
site are presented in a catalogue mode, 2) (2002) identified five classes of govern-
transaction (site uses database systems to ment web sites based on the service devel-

6
Evaluating E-Government

opment 1)emerging (simple online pres- perspectives and methodologies developed by

ence), 2)enhanced (information on the web academics and practitioners with regards to their
site becomes more dynamic) 3) interactive measurement and assessment.
(the web site enable two-way interactive This sections attempts to categorise both
communication through the web site) 4) quantitative and qualitative methodologies of
transactional (several transactions are con- governmental web sites’ evaluation. The concept of
ducted) 4) seamless (the site provides fully G2C and G2B web sites as web-based information
integrated digital services). systems is adopted in order to present the differ-
ent dimensions and aspects of their evaluation.
Based on the above classification, a local site Figure 4, illustrates the identified approaches of
(i.e. a Municipality’s web site) could be a trans- e-government web sites’ evaluation towards the
actional web site, whereas a national site (i.e. a basic elements and aspects of an integrated web-
Ministry’s portal) could be an enhanced site be- based information system delivering information
cause of the differences in the organisational and services to both business and citizens. In that
objectives or the differences in the technological context, governmental web site evaluation and ef-
requirements with regards to the services pro- fectiveness measurements attempt to investigate
vided by the government authorities. and assess its performance towards the strategic
and organisation objectives of the government
authority or agency operating the web site and the
EVALUATING E-GOVERNMENT end users’ needs and requirements using the web
WEB SITES site, in that case the outside stakeholders (citizens
and business interacting with the web site).
Web site evaluation refers to the assessment of a Specifically based on a literature review, the
web site’s performance towards specific criteria. following approaches and methodologies were
The importance of e-government web sites’ evalu- identified:
ation due to the vast investments of countries and
agencies and the raised requirements and expecta- • Conceptual evaluation frameworks’ imple-
tions of citizens for better online services, resulted mentation which identify the effectiveness
in a variety of models, methods and practices. With variables towards the web site’s objectives.
e-government web sites including a wide range of • Usability measurements to examine the
materials from simple information publications to user interface effectiveness.
databases and digital services available online, the • E-services’ qualitative measurements to
evaluation of governmental Internet performance assess the web site’s performance towards
has become a complex and demanding process, the delivery of digital services.
yet vital (Gupta & Jana, 2003). • User-satisfaction models to measure up the
Despite the importance of the evaluation overall positive or negative web site’s im-
procedure and the attention given by academics pact to its end users.
and practitioners, the evaluation of e-government
initiatives is considered to be immature with The evaluation measurement of e-government
reference to management and development tools web sites will provide meaningful feedback to
(Alshawi et al, 2007). In addition, because of the improve design, content and management features.
diversity in design and implementation of the The selection of the appropriate evaluation meth-
e-government web-based projects (Middleton, ods could create a scientific basis for governmental
2007), it is difficult to summarise the different administration decisions (Luo & Shi, 2010). Both

7
 Evaluating E-Government

Figure 4. Evaluation approaches of e-government web sites

subjective perceptions of the end users and web An early attempt to address the e-government
site consumers and measures based on objective evaluation in terms of citizens’ adoption by Carter
criteria are necessary to complete an overall and Belanger (2004) was based on constructs
analysis of the web site’s performance (Morgeson established in the area B2C e-commerce like Tech-
III & Mithas, 2009). nology Acceptance Model (TAM) and Diffusion
of Innovation Theory (DOI), but also introduced
Conceptual Evaluation Frameworks the perception of trustworthiness to encompass
the trust to Government dimension.
Conceptual frameworks in web site evaluation It is evident that many e-government evalu-
research identify the quality factors and their ation studies are influenced by research in the
interrelationships in order to investigate their im- area of Information Systems (IS) conceptualising
portance in terms of efficiency and effectiveness. e-government web sites as integrated information
With Regards to e-government, several evaluation systems delivering services to different stakehold-
frameworks have been implemented and tested ers in a complicated social and technical context.
by academics. This approach led to the study and analysis of
Table 3 presents some of the related research qualitative factors closely related to the IS theory
presenting the criteria examined and the method- like information, system and service quality (Wang
ologies used for their implementation or empirical et al, 2008) and the analysis of qualitative variables
testing. to address the social and financial aspects of the
An analysis of the conceptual evaluation e-government environment (Alshawi, 2007).
frameworks reveals that there is no identification Other approaches focus on the specific char-
or consistency with regards to their theoretical acteristics of the web site as the main medium
approach or the methodologies of assessment. to deliver information and services to citizens.
Each evaluation framework focuses on different Wang et al (2005) incorporated to their evalu-
aspects and elements of e-government web sites’ ation framework web site features based on the
development and operation. main citizens’ activities on government web sites,

8
Evaluating E-Government

Table 3. Conceptual evaluation frameworks for e-government web sites

Authors Concept Evaluation Criteria Methodologies

Carter and Provide a framework from the • DOI (compatibility, relative advantage, image, Data collected from
Belanger, 2004 integration of constructs from the complexity.) users.
technology • TAM (perceived ease of use, perceived
acceptance model (TAM), diffusions usefulness.)
of innovation • Trustworthiness (trust of Internet, trust of
theory (DOI) and Web trust model Government.)
to investigate the adoption of
e-government initiatives by the
citizens.
Web site evaluation is based on the Three clusters of variables: Case Study (web site
transaction between the individual • Site characteristic. development.)
Wang et al, 2005
& the web site. • Task characteristics.
• Individual characteristics.
Eschenfelder & Openness of e-government portals • Internal website/information characteristics. Comparative Case study
Miller, 2005 should be evaluated in a social- • Elements to capture the social and political of U.S state web sites.
technical context. context of the information.
• Assumptions about the roles of citizens and
government information.
Melitski et al, Provide an evaluation framework to • Security & privacy. Applied to several
2005 be applied national and international • Usability. e-government web sites.
e-government web sites. • Content.
• Services.
• Citizen participation.
Alshawi et al, Identify evaluating factors that • Technical issues (performance, accessibility.) Literature analysis
2007 influence citizens’ utilisation of • Economical issues (cost saving.)
e-government web sites. • Social issues (openness, trust, perceived ease
of use and perceived usefulness.
Liu et al, 2008 Provide Key Performance Indicators Value Categories: Comparative analysis
for different e-government • Strategic. of previous public value
stakeholders. • Operational. frameworks and Case
• Social. Study.
• Financial.
Wang and Liao, An adaption of an IS success model • Information quality. Data collected from
2008 in the e-government context. • System quality. users.
• Service quality.
• Use.
• User satisfaction.
• Perceived net benefit.

while Eschenfelder and Miller (2005) introduced Liu et al, (Liu, Derzsi, Raus & Kipp, 2008)
objective and subjective measures of the web were the first to provide Key Performance Area
site and its information to assess the openness of and Key Performance Indicators for government
government web sites and portals. In the same projects. Their work based on a comparative
concept, Melitski et al (2005) proposed factors analysis of previous value framework was em-
that influence the quality of government web sites pirical tested and provided four value categories
approach in a five-step framework addressing to assess the success of governmental web sites
usability, security, content, services and citizens with reference to their strategic role and objec-
participation issues. tives addressing financial and social implications
of their environment.

9
 Evaluating E-Government

On the whole, despite the limited and in some Usability measures apply to both commercial
ways unclear conceptual e-government evaluation and government web portals and sites and in most
research, the proposed frameworks introduced cases combine both classes of measures (Wood,
several key variables focusing on different aspects 2003). Specifically, for e-government projects,
and perspectives, providing guidance for the usability evaluation is of great significance due
development of both qualitative and quantitative to the heterogeneous Internet skills and the nature
evaluation indices. and the variety of the information and the digital
services delivered through the web site (Huang &
Usability Measures Brooks, 2011). Good usability enhances user satis-
faction and improves users’ trust in e-government
Web usability measures examine how useful, (Youngblood & Mackiewicz, 2012).
user-friendly and accessible is a web site with There are several usability evaluation measures
respect to the needs and expectations of its target of e-government web sites and portals. Differ-
audience. Simple, clear and consistent web site ent approaches identify and examine different
design enables users to perform their tasks and usability attributes and criteria. The following
activities effortlessly during their navigation table presents related usability studies (Table 4).
(Cappel & Juan, 2007). Nielsen (2000) created In most usability measures, the main evaluation
the acronym HOME (High quality – Often up- criteria are further analysed to specific guidelines
dated – Minimum Download time – Ease of use) with the use of usability heuristics which are either
to introduce the basic principles of usability that used by experts to perform an overall investigation
enhances the loyalty of a web site’s users. Also, the of the web site or transformed to tasks or typical
International ISO/IEC standard 9241-11 identifies scenarios executed by end-users (user testing)
three main attributes to define usability: (Huang, 2003; Huang & Brooks, 2011; Young-
blodd & Mackiwicz, 2012).
• Effectiveness A significant subarea of usability measures is
• Efficiency accessibility (Brajnik, 2000) which ‘’means that
• Satisfaction people with disabilities can perceive, understand,
navigate, and interact with the Web, and that they
These attributes are further analysed to create can contribute to the Web’’ (Wolrd Wide Web
specific usability metrics. Different approaches Consortium). Especially for government web
led to different sub criteria. Usability metrics sites, accessibility is a quality component that
are quality components that assess the site’s ef- should be considered as a requirement by ethics
fectiveness and efficiency in terms of information or in some countries by law and not an option
transparency, interactivity, design layout, aesthet- like in commercial web sites. For government
ics, accessibility, transactions (Casalo et al, 2005). organisations operating web sites it is essential to
Kappel et al (Kappel, Proll, Reiche & Retsch- provide equal opportunities to all citizens obtain-
itzegger) categorize usability measures in two ing information. Therefore, a lot of research in
classes: e-government evaluation has been focused on the
accessibility aspect applying specific guidelines
• User-based methods (i.e. user testing) are produced by the World Wide Web Consortium
based in the involvement and participation (W3C) (Huang, 2003; Abanumy, 2005; Baowaly
of end users in the evaluation process. et al, 2012). WCAG 1.0 is an internationally ac-
• Expert methods (i.e. heuristics evaluation) cepted standard including a set of 14 guidelines.
which require the experience of experts. Accessibility evaluations are conducted with the

10
Evaluating E-Government

Table 4. Usability studies for e-government web sites

Authors Criteria / Guidelines Evaluation object methodology

Katre & Gupta, 2011 • Accessibility State Government Web Portals Expert method
• Navigation
• Visual Design
• Information Content
• Branding
• Interactivity
• Ownership
Asiimwe & Lim, 2010 • Design Layout Ministries’ Web sites Expert method
• Navigation
• Legal Policies
Soufi & Maguire, 2007 • Information Architecture Local Government Web sites User based method
• Page Layout
• Design
• Accessibility
Baker, 2009 • Information Architecture Content analysis
• User Help
• Legitimacy
• Navigation
• Accessibility accommoda-
tions
• Online services

help of expert evaluators, the participation of vice quality has been well researched in the private
humans with disabilities or the use of specific sector and several models have been implemented
software tools and automated systems (Abanumy, to address the issue of service quality measure-
2005; Youngblood & Mackiwicz, 2012). ment. Research in the public sector has been
Scott (2005) identifies usability as one of the clearly affected by the conceptual approaches in
basic components that enhance the quality and the private sector, but the ‘’multiplicity’’ of the
ease of use of government web sites. Usability objectives and drivers of e-government organisa-
measures should be a continuous process in order tions requires the investigation of different dimen-
to identify mistakes in design and improve user sions (Buckley, 2003).
interface. Site’s usability could influence the adop- Based on literature review, qualitative measures
tion and engagement of citizens to government of e-government services can be categorised in
web sites and services. Government organisation two classes:
operating web sites should invest time and money
to improve their usefulness and quality. • The investigation of various qualitative
variables that influence the efficiency and
Quality Evaluation of effectiveness of the provided services in
E-Government Digital Services terms of information, communication and
transaction.
E-services in government web sites can be defined • The development of customer satisfac-
as the electronic delivery of information and ser- tion models which measure the perceived
vices. Specifically, services through government quality of services towards the users’
web sites describe the provision of information, expectations.
interactive communication and transactions Ser-

11
 Evaluating E-Government

E-Services Quality Evaluation Papadomichelaki et al (2006) propose four

key quality dimensions for e-government service
E-government services evaluation is based on quality evaluation:
the identification and testing of quality variables.
Kašubienė and Vanagas (2007) classified the • The service key area to assess the web sites
various quality criteria for e-services systems ability to deliver services accurately, con-
into three groups in order to point out the impact sistently and in time and facilitate interac-
of related research in the private sector to the tion between the sites’ stakeholders.
e-government area: • The content key area which involves qual-
ity dimensions relative to information and
• Criteria related to web sites (i.e. use, con- presentation aspects.
tent, structure, complexity.) • The system key area which includes quality
• Criteria related to features of traditional variables availability, accessibility, system
services (i.e. reliability, credibility, access, integrity, performance, reliability, interop-
ease of use, security.) erability, regulatory and security.
• Mixed criteria systems (i.e. web interac- • The organisation key area to address or-
tion, web interface, aesthetics.) ganisational perspectives.

Also, with respect to e-government, they also E-service quality in e-government has recently
divided e-service quality into information quality begun to receive increasing attention because of the
and process quality taking into account the dif- significant progress made in the development and
ferent stages of e-government systems’ maturity. implementation of digital services in government
On the whole, models in this class focuses on portal and web sites. The huge amounts invested
three core quality features information quality, in e-government projects and the uniqueness
process quality and service quality based on the of the services provided by local, regional and
main activities of citizens within the government national government web sites by citizens raises
web sites which is obtaining information, interact the importance of the theoretical grounding and
and proceed to transactions. the empirical research of e-government service
Research in this area led to the implementa- quality.
tion of several frameworks with reference to
models already introduced in the e-commerce User Satisfaction Models
area. Parasuraman (1988, 1991) theoretical model
SERVQUAL on traditional services was a great User satisfaction Index methodologies attempt
contribution to the development of e-government to quantify qualitative variables. Specifically,
service quality models. In this context, Viscusi quality dimensions of satisfaction are identified,
(2009) introduced eGovQual which incorporates weighted with regards to their impact on the
organisational, technical, social and juridical overall satisfaction and produce a final result –
perspectives to address issues like the diversity index. Data from users are required to select the
of the e-government stakeholders. Wang and variables and investigate their relationship to the
Liao (2008) proposed an IS success model in the overall consumer satisfaction. User satisfaction
e-government context identifying information models are cause-and-effect models which enable
quality, system quality, service quality, use, user to predict the effects of web sites’ changes and
satisfaction and perceived net benefit as significant enhancements to the overall satisfaction (Papa-
quality dimensions.

12
Evaluating E-Government

domichelaki, 2006) and project future behaviour Customer satisfaction index could be a sig-
based on satisfaction. nificant evaluation tool for countries to assess the
Such methodologies are commonly and widely efficiency and effectiveness of the implemented
used in the e-commerce area. In the e-government e-government web sites and portal. Improvements
research field, user/citizen satisfaction indices on measurement methodologies are necessary
have been developed and used in comparative (Abdelgawad and Snaprud, 2011). The quantitative
analyses and benchmarking researches to measure satisfaction metrics provide insight on the citizens’
the impact of e-government web sites. According perceptions towards specific quality dimensions
to Halaris et al (2007) customer satisfaction index of government e-services and identifies usability,
‘’is affected both from perceived by citizens quality functional or organisational problems and suggest
and from their expectations about the service’.’ improvements in this area. In an ideal world each
The American Satisfaction Index (ACSI) is a government would implement a national customer
leading and well established customer satisfac- satisfaction model to address the special charac-
tion model to evaluate both traditional and online teristics of e-government services consumers.
services. It was developed by the National Qual-
ity Research Centre and has served as basis for
the development of customer satisfaction index DESIGN AND EVALUATION
models in other countries. It is composed of six QUIDELINES AND STANDARDS
factors: perceived quality, customer expectations, BY THE U.S.A GOVERNMENT,
perceived value, overall customer satisfaction, THE EUROPEAN UNION AND THE
customer complaints, and customer loyalty. More UNITED NATIONS ORGANISATION
than 90 U.S. government web sites are evaluated
and rated quarterly by the ACSI. Results of each Due to the advanced progress and significance of
survey are announced to the public including e-government, evaluation procedures take place
information on the top rated e-government web in national and international level. Several assess-
sites, an overall satisfaction rate with e-government ment models were implemented to enhance cur-
services, suggestions on priorities to improve rent e-government initiatives and provide design
government web sites and comparative analysis guidelines for future e-government projects.
between satisfaction for the private and public E-government Act of 2002 was the first at-
sector. The European Consumer Satisfaction Index tempt to provide an e-government framework at
(EUSI) is another variation of the ACSI than can a national level by the government of the U.S.A.
be applied to companies in several industries and It’s objective was “to enhance the management
public organisations as well. and promotion of electronic Government services
Halaris et al (2007) identifies the following and processes by establishing a Federal Chief
principles/criteria of customer satisfaction models: Information Officer within the Office of Man-
agement and Budget, and by establishing a broad
• Service reliability framework of measures that require using Internet-
• Personalisation based information technology to enhance citizen
• Navigation/accessibility access to Government information and services,
• Information/content and for other purposes” (U.S. Government Print-
• Customer service ing Office). Under this act federal agencies were
obligated to utilise Internet in order to provide

13
 Evaluating E-Government

access to information and services to the public. • To facilitate citizens and businesses mobil-
In addition, standards were established to organise, ity in the single market.
preserve government information and make it ac- • Create a legitimate eUnion and gaining
cessible to citizens through federal web portals. high trust.
E-government Act of 2002, remains a prototype • Reduce administrative costs for citizens
of an e-government framework and was used as a and business.
basis for other similar models by other countries
or international organisations. Some of the indices introduced by this new
The United Nations (http://unpan3.un.org) framework are developments of broadband, ad-
introduced a comparative e-government evaluation vanced services, security, impact and investment
framework to support sustainable e-government in ICT research (Fitsilis et al, 2010). The European
development and also monitor the progress of Union announced already the new European e-
current e-government initiatives. The United government action plan 2011-2015 (ec.europe.
Nations annual global survey report presents “a eu). The objective of this new action plan is to
systematic evaluation of the use and potential of provide guidelines for more flexible, open and
ICT to transform the public sector by enhancing cooperative e-government services and at the same
efficiency, effectiveness, transparency, account- time increase the use of e-government services
ability, access to public services and citizen by 50% on households and 80% on businesses
participation in the all Member States of the by the year 2015.
United Nations, and at all levels of development” Over the last years, the United Nations and the
(The United Nations, 2013). The main features European Commission evaluation frameworks
reviewed by The United Nations survey are infor- provide suggestions for improvements in designing
mation dissemination/outreach, access/usability, web-based government services and present case
service delivery capability, citizen participation/ studies of successful e-government strategies.
interconnectedness. Some of the evaluation in-
dices adopted in The United Nations assessment
model measure e-government readiness, e-service CONCLUSION
delivery, telecommunication infrastructure index,
e-participation indices. The United Nations vision E-government progresses vastly in many coun-
for e-government is Connected Governance which tries. Web portals and web sites remain the main
will enhance public administration efficiency and and most significant component of e-government
public service delivery. construction. Enormous amounts of money are
Similarly, the European Union and specifi- invested for their implementation by local, re-
cally the European Commission investigates the gional and national agencies and authorities. In
progress of e-government initiatives in the Eu- most cases, governmental web sites integrate G2C
ropean Countries. Multiple surveys with the use and G2B policies. With their development being
of benchmarking methodologies are conducted a continuous process, evaluation measurements
to provide an insight on the development and are vital for their optimisation in delivering criti-
use of ICT for interacting with public adminis- cal information and unique services. A thorough
trations in the European Union. Currently, the literature review identified several measurement
i2010 framework which follows previous action approaches focusing on different dimensions
plans of the European Union (e-Europe 2003 and and features of the government web sites design
e-Europe 2005) sets three political priorities for and functionality. Based on the concept that a
e-government: government web site is an integrated web-based

14
Evaluating E-Government

information system, with citizens and enterprises Al-adawi, Z., Yousafzai, S., & Pallister, J.
being its end-users, evaluation measurements are (2005). Conceptual model of citizen adoption
categorised in four classes: 1) conceptual evalu- of e-government. In Proceedings of the Second
ation frameworks, 2) usability measurements, 3) International Conference on Innovations in Infor-
e-services’ quality measurements and 4) customer mation Technology. Retrieved April 10, 2013, from
satisfaction models. http://www.it-innovations.ae/iit005/proceedings/
So far, the empirical testing of the several articles/G_6_IIT05-Al-Adawi.pdf
e-government evaluation models revealed a gap
Almarabeh, T., & Abu Ali, A. (2010). A general
between theory and practice. In most cases,
framework for e-government: Definition, maturity,
government web sites are in their early maturity
challenges, opportunities and success. European
phases (simple online presence or interactive web
Journal of Scientific Research, 39, 29–42.
sites) or having major usability and accessibility
problems. Improvements are required to meet the Asiimwe, E. N., & Lim, N. (2010). Usability
raised expectations of citizens and businesses. of government websites in Uganda. Electronic.
Little attention has been given by govern- Journal of E-Government, 8(1), 1–12.
ments on the evaluation stage of a web sites life
Baker, D. L. (2009). Advancing e-government
cycle. However, a lot of surveys conducted in an
performance in the United States through en-
international or supranational level by the United
hanced usability benchmarks. Government Infor-
Nations and the European Union, provide guid-
mation Quarterly, 26(1), 82–88. doi:10.1016/j.
ance and suggestions for better services design
giq.2008.01.004
and web-based system’s implementation.
E-government is now facing a new era. Web 2.0 Bannister, F. (2007). The curse of the benchmark:
provides new opportunities for governments (Chua An assessment of the validity and value of e-
et al, 2012) and will cause a profound change on the government comparisons. International Review
delivery of information and services. Governments of Administrative Sciences, 73(2), 171–188.
should be devoted to utilise and measure effective doi:10.1177/0020852307077959
use of Internet enabled technologies. A thorough
Baowaly, M., Hossain, M., & Bhuiyan, M.
investigation and evaluation of an e-government
(2012). Accessibility analysis and evaluation for
web site should be considered an investment to
government-websites in developing countries:
achieve efficiency and effectiveness.
Case study Bangladesh. Computer Engineering
and Intelligent Systems, 3(4).
REFERENCES Baum, C., & Di Maio, A. (2000). Gartner’s four
phases of e-government model. Retrieved April
W3C. (2013). Accessibility — W3C. Retrieved 15, 2013, from http://www.gartner.com
April 20, 2013, from, http://www.w3.org/stan-
dards/webdesign/accessibility Beynon-Davies, P. (2007). Models for e-govern-
ment. Transforming Government: People. Process
Abanumy, A., Al-Badi, A., & Mayhew, P. (2005). and Policy, 1(1), 7–28.
e-Government website accessibility: In-depth
evaluation of Saudi Arabia and Oman. The Elec- Brajnik, G. (2000). Automatic web usability
tronic. Journal of E-Government, 3(3), 99–106. evaluation: What needs to be done? In Proceed-
ings of the 6th Conference on Human Factors &
the Web. IEEE.

15
 Evaluating E-Government

Cappel, J., & Huang, Z. (2007). A usability analy- Fang, Z. (2002). E-government in the digital era:
sis of company websites. Journal of Computer Concept, practice and development. International
Information Systems, 48(1), 117–123. Journal of The computer. The Internet and Man-
agement, 10(20), 1–22.
Carter, L., & Belanger, F. (2004). Citizen adoption
of electronic government initiatives. In Proceed- Gupta, M. P., & Jana, D. (2003). E-government
ings of 37th Annual Hawaii International Confer- evaluation: A framework and case study. Gov-
ence on System Sciences. IEEE. ernment Information Quarterly, 20, 365–387.
doi:10.1016/j.giq.2003.08.002
Carter, L., & Belanger, F. (2005). The utiliza-
tion of e-government services: Citizen trust, Halaris, C., Magoutas, B., Papadomichelaki,
innovation and acceptance factors. Information X., & Mentzas, G. (2007). Classification and
Systems Journal, 15(1), 5–25. doi:10.1111/j.1365- synthesis of quality approaches in e-government
2575.2005.00183.x services. Internet Research: Electronic Network-
ing Applications and Policy, 17(4), 378–401.
Casaló, L., Flavián, C., & Guinaliú, M. (2005).
doi:10.1108/10662240710828058
The role of accessibility and commitment in
the development of an e-government strategy. Holmes, D. (2001). E-gov: E-business strate-
In Proceedings of eGovernment Workshop ‘05 gies for government. London: Nicholas Brealey
(eGOV05). Brunel University. Publishing.
Chua, A., Goh, D., & Ang, R. (2012). Web 2.0 Huang, J. C. (2003). Usability of e-government
applications in government websites: Prevalence, web-sites for people with disabilities. In Proceed-
use and correlations with perceived web site qual- ings of the 36th Hawaii International Conference
ity. Online Information Review, 36(2), 175–195. on System Sciences. IEEE.
doi:10.1108/14684521211229020
Huang, Z., & Brooks, L. (2011). Credibility and
Coursey, D., & Norris, D. F. (2008). Models of usability evaluation of e-governments: Heuristic
e-government: Are they correct? An empirical evaluation approach. In Proceedings of tGov 2011.
assessment. Public Administration Review, 68(3), Brunel University.
523–536. doi:10.1111/j.1540-6210.2008.00888.x
Jansen, A., & lnes, S. (2004). Quality assessment
Eschenfelder, K. R., & Miller, C. (2005). The and benchmarking of Norwegian public web sites.
openness of government websites: Toward a In Proceeding from European Conference on
socio-technical government website evaluation E-Government. Retrieved March 15, 2013, from
toolkit. Seattle, WA: MacArthur Foundation/ALA http://www.afin.uio.no/english/research/ arild/
Office of Information Technology Policy Internet QualityAssessment.pdf
Credibility and the User Symposium.
Jeong Chun Hai @Ibrahim. (2007). Fundamental
Fairweather, N., & Rogerson, S. (2006). Towards of development administration. Selangor: Scholar
morally defensible e-government interactions Press.
with citizens. Journal of Information. Commu-
Jorgenson, D., & Cable, S. (2002, Summer). Facing
nication and Ethics in Society, 4(4), 173–180.
the challenges of e-government: A case study of
doi:10.1108/14779960680000290
the city of Corpus Christi, Texas. SAM Advanced
Management Journal.

16
Evaluating E-Government

Kappel, G., Proll, B., Reich, S., & Retschitzeg- Palvia, S. C. J., & Sharma, S. S. (2007). E-
ger, W. (2006). Web engineering: The discipline government and e-governance: Definitions/
of systematic development of web applications. domain framework and status around the world.
London: John Wiley & Sons Ltd. In Foundation of e-government. ICEG.
Kašubienė, L., & Vanagas, P. (2007). Assumptions Papadomichelaki, X., Magoutas, B., Halaris, C.,
of e-government services quality evaluation. The Apostolou, D., & Mentzas, G. (2006). A review
Engineering Economist, 5(55), 68–74. of quality dimensions in e-government services.
In M. Wimmer et al. (Eds.), EGOV 2006 (LNCS)
Katre, D. S., & Gupta, M. (2011). Expert usability
(Vol. 4084, pp. 128–138). Berlin: Springer.
evaluation of 28 state government web portals of
doi:10.1007/11823100_12
India. International Journal of Public Information
Systems, (3), 115-130. Parasuraman, A., Berry, L. L., & Zeithaml, V. A.
(1988). SERVQUAL: A multiple-item scale for
Layne, K., & Lee, J. (2001). Developing fully
measuring customer perceptions of service quality.
functional e-government: A four stagemodel. Gov-
Journal of Retailing, 64(1), 12–40.
ernment Information Quarterly, 18(2), 122–136.
doi:10.1016/S0740-624X(01)00066-1 Parasuraman, A., Berry, L. L., & Zeithaml, V.
A. (1991). Refinement and reassessment of the
Liu, J., Derzsi, Z., Raus, M., & Kipp, A. (2008).
SERVQUAL scale. Journal of Retailing, 67(4),
eGovernment project evaluation: An integrated
420–450.
framework. Lecture Notes in Computer Science,
5184, 85–97. doi:10.1007/978-3-540-85204-9_8 Reddick, C. (2011). Citizen interaction and e-
government: Evidence for the managerial, con-
Melitski, J., Holzer, M., Kim, S.-T., Kim, C.-G.,
sultative, and participatory models. Transforming
& Rho, S.-Y. (2005). Digital government world-
Government: People. Process and Policy, 5(2),
wide: An e-government assessment of municipal
167–184.
web sites. International Journal of Electronic
Government Research, 1(1), 1–19. doi:10.4018/ Scholl, H. J., Barzilai-Nahon, K., Jin-Hyuk, A.,
jegr.2005010101 Popova, O. H., & Re, B. (2009). E-commerce
and e-government: How do they compare? What
Middleton, M. R. (2007). Approaches to evalu-
can they learn from each other? In Proceedings
ation of websites for public sector services. In
of System Sciences, 42nd Hawaii International
Proceedings IADIS Conference on e-Society, (pp.
Conference on system Sciences. IEEE.
279-284). Lisbon, Portugal: IADIS.
Scott, J. K. (2005). Assessing the qual-
Morgeson, F. V. III, & Mithas, S. (2009). Does e-
ity of municipal government websites. State
government measure up to e-business? Comparing
& Local Government Review, 37(2), 151–165.
end-user perceptions of U.S. federal government
doi:10.1177/0160323X0503700206
and e-business websites. Public Administra-
tion Review, 69, 740–752. doi:10.1111/j.1540- Soufi, B., & Maguire, B. (2007). Achieving us-
6210.2009.02021.x ability within e-government web sites illustrated
by a case study evaluation. In Human interface
Nielsen, J. (2000). Designing web usability: The
and the management of information: Interacting
practice of simplicity. New Riders Publishing.
in information environments (LNCS) (Vol. 4558,
pp. 777–784). Berlin: Springer. doi:10.1007/978-
3-540-73354-6_85

17
 Evaluating E-Government

Stahl, B.C. (2005). The paradigm of e-commerce Youngblood, N., & Mackiewicz, J. (2012, June).
in e-government and e-democracy. Electronic A usability analysis of municipal government
Government Strategies and Implementation, 1 -19. website home pages in Alabama. Government
Information Quarterly, 582–588. doi:10.1016/j.
United Nations. (2008). UN e-government survey
giq.2011.12.010
2008: From e-government to connected gover-
nance. Retrieved May 12, 2013, from http://www.
ansa-africa.net/uploads/documents/publications/
UN_e-government_survey_2008.pdf KEY TERMS AND DEFINITIONS
United Nations. (2011). E-government. Retrieved E-Government: The use of Information and
March 10, 2013, from http://unpan1.un.org Communication Technology (ICT) and particu-
Viscusi, G. (2009). The eGovQual methodol- larly Internet-based and Web-based telecommu-
ogy: Information systems planning as research nication practices to facilitate connections within
intervention. Working Papers on Information the authorities and also deliver digital services to
Systems, 9(15). citizens and companies.
E-Government Business Models: Identify the
Wang, L., Bretschneider, S., & Gant, J. (2005). key stakeholders in the e-government environment
Evaluating web-based e-government services with and their interrelationships and specify the key
a citizen-centric approach. In Proceedings of the business processes that represent the integration
38th Hawaii International Conference on System of governmental information and services.
Sciences, (pp. 129-137). IEEE. E-Government Web Site Evaluation: Is the
Wang, Y. S., & Liao, Y. W. (2008). Assessing assessment of a governmental web site towards
egovernment systems success: A validation of the specific criteria.
DeLone and McLean model of information sys- E-Government Web Site Typology: A cat-
tems success. Government Information Quarterly, egorisation of governmental portals and web sites
25(4), 717–733. doi:10.1016/j.giq.2007.06.002 based on the level of e-government and their level
of growth (maturity)/service development.
Warkentin, M., Gefen, D., Pavlou, P. A., E-Services Quality Evaluation: Is the
& Rose, G. M. (2002). Encouraging citi- identification and testing of quality variables for
zen adoption of e-governement by building e-services systems.
trust. Electronic Markets, 72(3), 157–162. Usability Measures: Examine how useful,
doi:10.1080/101967802320245929 user-friendly and accessible is a governmental web
Wood, F. B., Siegel, E. R., LaCroix, E., & Lyon, site with respect to the needs and expectations of
B. (2003). A practical approach to e-government its target audience.
web evaluation. In IT professional human interface User Satisfaction Models: Are cause-effect
and the management of information: Interacting models which enable the prediction of effects to
in information environments (LNCS) (Vol. 4558, governmental web sites’ changes to the overall
pp. 777–784). Berlin: Springer. doi:10.1109/ citizen satisfaction.
MITP.2003.1202231
World Bank. (2013). Defining e-government.
Retrieved March 10, 2013 from http://web.
worldbank.org

18

View publication stats