2012 Sandoval Almazan
2012 Sandoval Almazan
2012 Sandoval Almazan
a r t i c l e i n f o a b s t r a c t
Available online 27 November 2011 Scholars and practitioners argue that the most important interactions between citizens and government hap-
pen at the local level. These relationships could become closer and more frequent with the use of information
Keywords: and communication technologies (ICTs). In fact, portals could be seen not only as channels for providing
E-government government information and services, but also as powerful tools to exchange information and knowledge
Municipalities between different social actors and government entities and to enable participation in collective decision-
Local e-government
making efforts about important public affairs. For instance, social media and other Web 2.0 tools could pro-
E-governance
Local government portals
vide new electronic channels for these interactions through their inclusion in local government portals. This
E-government websites paper argues that although important modifications to the organizational and institutional frameworks
Internet portals would be necessary, the potential for local electronic governance through networks of government and
non-government actors via internet portals is clearly present. However, a very important first step would
be the inclusion of more interaction, participation, and collaboration mechanisms in government portals.
Similar to previous efforts with data from the U.S., this paper describes the results of a recent assessment
of local government portals in Mexico. The conclusion is that progress toward citizen engagement is slow
in local governments and there are very few efforts to increase interaction, participation, and collaboration
channels on their portals. Most of them are still following the vision of information and services providers
and a local electronic governance model is still in its very initial stages. It seems that e-government in munic-
ipalities is still more rhetoric and less reality, at least in some countries.
© 2011 Elsevier Inc. All rights reserved.
1. Introduction of new platforms such as mobile devices and social media expand the
possibilities to interact, participate, and collaborate among government
The use of information and communication technologies (ICTs) in agencies and between government and non-government actors, creat-
government has significantly increased in the last few decades. Coun- ing the potential for a new local e-governance model.
tries around the world are now adopting strategies for better use of Before talking about e-government in municipalities, it is necessary
these technologies with very different objectives: greater efficiency, to have a conception of the e-government phenomenon. In this respect,
deeper transparency, higher service quality, and more engaged citizen there is a dynamic debate about the concept of electronic government.
participation. What is now called “electronic government” has become For the purpose of this research, we consider several diverse definitions.
a powerful strategy for administrative reform at all levels of govern- For example, Deloitte (2000) proposes a fairly strict definition of
ment. However, federal and state governments are more clearly taking e-government as “the use of technology to enhance the access to
advantage of the potential benefits of new and emergent information and delivery of government services to benefit citizens, business part-
technologies. Local governments are also using ICTs, but there are rela- ners and employees.” In contrast, Gil-Garcia and Luna-Reyes (2006)
tively few studies about local e-government, particularly in developing contend that “electronic government is the selection, implementation,
countries, and more knowledge should be generated. In addition, the and use of information and communication technologies in government
use of information technologies in local governments is emerging; to provide public services, improve managerial effectiveness, and pro-
most of them use ICTs to display information and provide some services, mote democratic values and mechanisms, as well as the development
but very few have used them for promoting greater participation and of a regulatory framework that facilitates information-intensive initia-
collaboration. However, the introduction of Web 2.0 tools and the use tives and fosters the knowledge society” (p. 639). This definition allows
us to consider front and back office processes and multiple aspects of
the use of IT in government. In addition, this definition implicitly in-
⁎ Corresponding author. cludes some important elements such as participation, collaboration,
E-mail address: rsandovuaem@gmail.com (R. Sandoval-Almazan). transparency, and modifications to the legal framework.
0740-624X/$ – see front matter © 2011 Elsevier Inc. All rights reserved.
doi:10.1016/j.giq.2011.09.004
R. Sandoval-Almazan, J.R. Gil-Garcia / Government Information Quarterly 29 (2012) S72–S81 S73
State and local governments are expending large amounts of money insights about the services local governments provide and the use of
to introduce ICTs in both internal operations and the provision of public IT for developing their websites (Kaylor, Deshazo, & Van Eck, 2001).
services. Many of these governments have created Web pages, which pro- Some previous studies compare information sharing practices of federal
vide information about government agencies and, in some cases, allow for agencies and local governments (Bochicchio, Celentano, & Longo, 2004;
online transactions and interactions with citizens (Gant & Gant, 2002). Leenes & Svensson, 2002). Moon's (2002) article became the founda-
Other local governments around the world have been using IT in their tion for research on the evolution of local e-government. In this article,
back-office processes for a long time. In fact, some advanced applications he suggests that in reality local governments were clearly behind the
in public services, websites, human resources, collaboration and, most re- potential results and expectations generated regarding the use of infor-
cently, e-democracy are also present at the local level. In addition, Web mation technologies. More recently, Reddick (2004) proposed a two-
2.0 tools and other emergent technologies could easily be integrated stage model of local e-government growth. The first stage includes
with existing government portals and potentially expand and enrich the data catalogs only, while the second stage has online transactions. Sim-
relationships and exchanges of information among government agencies ilar examples of county or municipal surveys have been developed in
and between government agencies and citizens, businesses, non-for- many countries around the world (Al-Nuaim, 2009; Attour-Oueslati,
profit organizations, other levels of government, and other government Dufresne, & Longhi, 2007; Baker, Hanson, & Myhill, 2009; Chatzopoulos
branches. Moon (2002) suggested that more interaction and participation & Economides, 2009; Evans-Cowley, 2005; Hahamis, Iles, & Healy,
would lead to obtain many of the expected outcomes, making the rhetoric 2005; Kunstelj & Decman, 2005; Williams, 2008). There are also a few
a reality. It seems that some of these tools and applications could help to worldwide studies of local government websites (Melitski, Holzer,
make this transition faster than originally expected. Kim, Kim, & Rho, 2005; Rodríguez, Welicki, Giulianelli, & Vera, 2008).
This paper proposes that government websites could become central Many of these studies attempt to assess the degree of technical and or-
components of public information sharing networks, which potentially ganizational sophistication of e-government at the local level.
include government and non-government actors communicating, partic- Regarding websites, Gant and Gant (2002) state, “A Web portal
ipating, collaborating, and sharing information. In addition, these net- serves as the integrated gateway into the state government web sites
works could be the basis for new local e-governance models, since and provides visitors with a single point of contact for online service de-
there is the potential for true collaboration and joint decision-making. livery within the state.” These portals contain an interesting mixture of
However, a very important first step would be the inclusion of more in- applications and are good examples of current e-government efforts
teraction, participation, and collaboration mechanisms in government (Sandoval-Almazan & Gil-Garcia, 2008a). On the other hand, e-
portals. The paper is divided in five sections, including the foregoing in- government portals have evolved during the last few years, from unidi-
troduction. The second section presents a brief review of recent literature rectional sharing of information and data with citizens to more interac-
about local e-government, internet portals, and their relationships with tive engagement. Recent technological innovations have, at least in part,
communication, interaction, participation, and information sharing. Sec- caused this evolution, such as Web 2.0 applications and tools. Some mu-
tion three describes the research design and methods, including aspects nicipalities are using these technologies, but they lack a clear strategy,
of the website evaluation instrument and the interviews. Section four which can cause results that are not necessarily as expected
presents the main findings of this study in terms of the proposed con- (Sandoval-Almazan & Gil-Garcia, 2009a). E-government portals need
cepts. Finally, section five provides some final comments and suggests to continuously evolve and adapt to new internet features. This evolu-
areas for future research within this topic. tion has also transformed the main functions that websites can perform
and the way they relate to citizens and other social actors (Lips, 2010).
2. Government portals, participation, and collaboration Government websites can be considered communication systems
via a computer and the internet (Sandoval-Almazan & Gil-Garcia,
Two key issues facing the implementation of electronic government 2010). There are different models that explain the development and
at all levels of government are collaboration and information sharing. In evolution of government websites (Gil-Garcia & Martinez-Moyano,
addition, the current changing environment, new decision-making pro- 2007; Layne & Lee, 2001; Sandoval-Almazan & Gil-Garcia, 2008a,
cedures, and the complexity of the problems they face make the flow of 2010). One of the most frequently used approaches is the evolution-
information and the exchange of data important challenges. With the ary perspective, which creates stages and analyzes e-government ini-
goal of better understanding this problem, scholars have developed dif- tiatives according to the characteristics and technical features found
ferent methodologies and models of cross-boundary information shar- in these stages (i.e., presence, interaction, integration). The evolution-
ing and integration at the local, national, and international levels ary approach is useful to understand government websites because it
(Dawes, 2010; Gant, 2003; Gelders, Brans, Maesschalck, & Colsoul, attempts to measure the degree of innovation and provides guidance
2010; Klievink & Janssen, 2010; Navarrete, Gil-Garcia, Mellouli, Pardo, for developments and improvements (Layne & Lee, 2001; Moon,
& Scholl, 2010; Papenfuß & Schaefer, 2010; Williams et al., 2009; 2002; Sandoval-Almazan & Gil-Garcia, 2006).
Zheng, Yang, Pardo, & Jiang, 2009). In fact, as early as 1996, Dawes de- This evolutionary approach includes the assumption that there is,
scribed the benefits and challenges of information sharing initiatives in fact, an evolution towards electronic government (Gil-Garcia &
(Dawes, 1996). Information sharing can take place using a great variety Martinez-Moyano, 2007). Some authors contend that each one of
of information technologies. This article argues that government portals the stages is already electronic government. Others delimit the
could be a central technical component for interaction, participation, phases in which a government can be considered electronic. Recent
and collaboration within networks of government and non- studies demonstrate that evolutionary approaches are useful to un-
government actors. The section that follows presents previous studies derstand and evaluate e-government, but have important limitations
about local e-government, including some important concepts related (Gottschalk, 2009; Sandoval-Almazan & Gil-Garcia, 2008a, 2008b,
to different functions performed by government websites. The section 2009b). In addition, evaluating e-government initiatives, including
also briefly talks about public information sharing networks, as a useful web portals, using this approach could be difficult because they can
concept to understand the relationships between multiple social actors have characteristics and features identified with multiple stages. In-
in collaboration settings. formation, interaction, transaction, integration, and participation
have been proposed as complementary components, not necessarily
2.1. Local electronic government and functions of internet portals mutually exclusive and consecutive stages (Sandoval-Almazan &
Gil-Garcia, 2008b). These components could be used to characterize
One of the first international studies about e-government at the local the development of government websites and other e-government
level was developed in the United States and provided some initial initiatives (Sandoval-Almazan & Gil-Garcia, 2010). Fig. 1 presents a
S74 R. Sandoval-Almazan, J.R. Gil-Garcia / Government Information Quarterly 29 (2012) S72–S81
Opportunities
Information Provision of Tools for Channels for
for
Display Services Interaction Participation
Collaboration
variation on these components, which position them as functions in a create new connections, and enable public officials to make better deci-
continuum from less collaboration and participation to more collabo- sions with more complete information (Ballejos & Montagna, 2010).
ration and participation. Following, we explain each of the functions Other tools beyond traditional email and intranets could be used to pro-
and some of the government website applications related to them. mote internal knowledge sharing (Noveck, 2009), including collabora-
tive wikis, mind mapping, or the use of RSS and forums. In the
2.1.1. Information display external direction, the e-government website could be a starting point
A basic function of e-government portals is the display of public in- for several interactions with citizens, like the use of wikis to share
formation (Kun Chang, Melih, Sangjae, & Gyoo Gun, 2008; Susan, 2004). ideas and solutions or promote orderly discussion about public policies
The presentation format of government information is important in (Noveck, 2009; Tapscott & Wiliams, 2010). Another interaction channel
terms of the degree of success of a website (Susan, 2004). Some portals is the use of blogging by public officials in the major offices to promote
use traditional structures such as directories or menus, others allow initiatives, introduce new ideas, or simply to communicate with
users to search for information using a search engine, and more recently citizens.
some of them use multimedia tools such as video, audio, or online pre- Another way to interact is an official government Twitter account
sentations (Dobrev, Stoewer, Makris, & Getsova, 2002). Despite these that is used to send messages with news, warnings, or novel ideas
differences, it seems that they all attempt to provide valuable data from the municipality. The use of groups on Facebook could connect
and information to citizens. Recently, Buccoliero and Bellio (2010) ana- citizens with public officials or other citizens in the same city to
lyzed Venice's use of Web 2.0 projects in order to define the content that share problems or discuss public policies. Interaction with citizens
an efficient web strategy should include. They found that information could also happen without the use of Web 2.0 tools; for example, gov-
display should be directly related with the back-office structures of ernments can incorporate comment boxes within the portals to gath-
the local government. The provision of updated and accurate informa- er opinions or host an online chat with different public officials based
tion is linked with internal efficiency and an organizational design to on specific topics. The key is to introduce these tools on the website
ensure the information is up-to-date (Gould, Gomez, & Camacho, with a clear strategy and a general perspective of fostering interaction
2010). However, usually local governments use the website to have a between citizens and government.
local presence online, but not as part of an overarching government
strategy to link with citizens (Lips, 2010). 2.1.4. Channels for participation
Citizens participate with government through several channels.
2.1.2. Provision of services These channels vary from participation in the democratic process to
The provision of services on e-government portals has become a the use of communication channels to acquire services, submit
key differentiator among them. Government websites that introduce claims, and gain access to public officials (Anthopoulos, Siozos, &
online services could be considered successful; however, there Tsoukalas, 2007; Boyer-Wright & Kottemann, 2008). The idea of a
might be few services that can be performed totally online. For exam- multi-channel strategy for improving citizen participation is not
ple, Gouscos, Mentzas, and Georgiadis (2001) discussed a system for new; Ebbers, Pieterson, and Noordman (2008) propose this strategy
the provision of services to companies that require information to expand the opportunities for citizens to exchange data and knowl-
about transactions on cross-border issues. However, most of these edge. Gil-Garcia, Chun, and Janssen (2009) argue that intermediaries –
services require additional steps and/or a physical presence in a gov- who could be citizens – can add value and information quality to
ernment office. In addition, citizen expectations about online services services. Participation could create value for citizens and government
are continually increasing (Gould et al., 2010). Recently, local govern- organizations alike.
ments have become more aware of the limitations to their services Research also indicates that e-government websites could be the
and are trying to improve them (Connolly, Bannister, & Kearney, gateway to electronic channels for participation (Barnes & Vidgen,
2010; Papenfuß & Schaefer, 2010; Paris, 2006). Research has also sug- 2007). Through the use of online surveys, forums, and other applica-
gested that there are some alternatives to government web portals tions, participation and interaction with government officials and
for introducing electronic services, such as mobile phones (Ntaliani, agencies has expanded (Ong & Wang, 2009). However, the regular
Costopoulou, Manouselis, & Karetsos, 2009). use and update of these channels is important for citizens. Lack of ac-
tivity and inaccurate or outdated information may cause citizens to
2.1.3. Tools for interaction avoid the government portals (Fink, 2010; Resca, 2010). Finally, the
State and local government websites have started using Web 2.0 use of electronic channels can be improved with an online strategy
tools and applications, but very few of them establish a clear interaction that personalizes government information for citizens, based on de-
strategy (Sandoval-Almazan, Diaz-Murillo, Gil-Garcia, & Luna-Reyes, tails like location or demographics, to provide better information for
2010). Interaction through the e-government website could be ana- individual decision making.
lyzed in two directions. The internal direction is when the website al-
lows interaction among agencies, public officials, and operative 2.1.5. Opportunities for collaboration
personnel of the local government. This interaction could be restricted, Collaboration is the last component and a desirable one for e-
private, or focus on specialized groups. Many of the available tools, like government websites. Usually citizens access government websites
blogs and social media (Twitter and Facebook), can be used to promote to find information and data for decision making; very few explicitly
internal participation in a structured fashion and improve relationships, want to collaborate with government. Collaboration results from a
R. Sandoval-Almazan, J.R. Gil-Garcia / Government Information Quarterly 29 (2012) S72–S81 S75
long relationship that produces trust. Citizens around the world have and facilitates collaboration. Dawes et al. (2009) explain the difference
consistently shown low levels of trust in government (Inglehart, between a network and a public network: “Unlike other types of net-
1997; Parent, Vandebeek, & Gemino, 2005; Tolbert & Mossberger, works, Public Sector Knowledge Networks (PSKNs) treat information
2006). However, applications such as wikis, Facebook, Twitter, and and knowledge sharing across traditional organizational boundaries as
even the normal use of email have created different forms of interac- a primary purpose as they try to address public needs that no single or-
tion and increasingly produce more collaboration. For example, the ganization or jurisdiction can handle alone” (p. 392). This kind of net-
model to promote a participatory budget in Brazil is an important work has bidirectional relationships with people, processes, software,
case to understand online collaboration as a way to connect citizen and other information technologies (Dawes et al., 2009).
needs and local government issues (Matheus & Ribeiro, 2009). The We propose that government websites or internet portals could play
cases of Porto Alegre, Ipatinga, Belo Horizonte, and Recife in Brazil a very important role in local governance models that use the principles
and Miraflores in Peru present evidence of collaboration; they rely of PSKNs. Most government data converges in the e-government web-
on high quality information and access to information technologies sites to display information, provide services, and interact with citizens.
(Matheus, Ribeiro, Vaz, & Souza, 2010). They could also include tools and applications for increasing participa-
Collaboration can also include NGOs, other agencies, and even the tion, collaboration, and information sharing. Following, we briefly ex-
media – newspapers, television, or radio – that need to interact with plain the processes of communication, engagement, and information
government. Sometimes collaboration rises during emergencies, sharing and how they relate to specific tools for interaction, channels
when government portals provide citizens with several channels to for participation, and opportunities for collaboration.
access information for quick decision making and enables both
parties – citizens and government – to collaborate in solving the 2.2.1. Communication and tools for interaction
emergency (Maldonado, Maitland, & Tapia, 2010). Tapscott and Web 2.0 applications may be considered technologies for commu-
Wiliams (2010) assert that the new role of citizens in a connected nication. Some of these applications are the so-called social networks,
world is to be a prosumer: a producer and consumer of information micro-formats, social labels, RSS (content syndication), blogs, video-
at the same time. They propose that intensive collaboration could blogs, podcasts, wikis and forums, etc. Examples of commercial web-
help governments face urgent problems, like the 2008 financial crisis sites that implement these applications include Technorati, Digg,
and ongoing environmental crises. Facebook, Flickr, YouTube, MySpace, Twitter and Del.icio.us, amongst
others. Some government sites are also starting to include some of
2.2. Communication, engagement, and information sharing these applications (Cassell & Hoornbeek, 2010). For instance, recent
research shows that Mexican local government portals are using
After the terrorist attacks in the United States on September 11th, these tools, although in a very limited fashion (Sandoval-Almazan &
information sharing was widely recognized as an important subject. Castaneda, 2010). Different tools are used for different purposes on
Several scholars argue for the urgency to develop new ideas and models government portals. Table 1 shows some examples of the use of social
to improve information sharing among government agencies (Clarkson, media applications on government websites.
Jacobsen, & Batcheller, 2007; Evans-Cowley, 2005; Lee & Rao, 2007; Social media has become very important for communication
White & DiCenso, 2005). There are some recent studies that attempt among politicians, citizens, and public managers (Sandoval-Almazan
to understand inter-organizational collaboration and information shar- & Castaneda, 2010; Sandoval-Almazan & Gil-Garcia, 2010; Wigand,
ing in different contexts (Canestraro, Pardo, Raup-Kounovsky, & 2010). The use of Twitter and Facebook to enable contact with citi-
Taratus, 2009; Navarrete, Mellouli, Pardo, & Gil-Garcia, 2009; Navarrete zens is now a normal practice in some e-government portals. What
et al., 2010; Pardo, Gil-Garcia, & Burke, 2008). We argue that government needs to be measured is the impact these tools have on public policy
portals could help to facilitate interaction, participation, and collabora- decisions, political protest over the web, or improving government
tion among multiple government and non-government actors (see operations and reducing costs (Gelders et al., 2010).
Fig. 2). They could become key components of public sector knowledge
networks and, in the long run, enable a new local e-governance model. 2.2.2. Engagement and channels for participation
For some scholars, research about collaboration and information The most common participation channels on government portals
sharing should focus on government processes and new organizational are surveys and forums. These unidirectional tools enable citizens to
structures and rules (Gil-Garcia et al., 2009). In order to achieve this engage with government by giving their opinion 24/7 about policies,
goal, a new organizational paradigm has been proposed: Public Sector political statements, decisions, or public service malfunctions. How-
Knowledge Networks (Dawes, Cresswell, & Pardo, 2009). Information ever, the problem with these channels is the lack of feedback from
exchange and sharing using networks enhances government processes the government side; many times government officials are unaware
of the forums or surveys results and only take part in the initial mar-
keting strategy. On the other hand, the use of email on the websites
Individual can enable more engaged and personalized connections between cit-
Citizens izens and public servants on a regular basis (Matheus & Ribeiro,
Municipality
NGOs 2009). A simple email directory allows citizens to directly contact
Council
the person involved in their problem or need (Ong & Wang, 2009).
Another participation channel is a blog. This tool provides a human
side to the websites and promotes citizen engagement because they
Private
Mayor Portal Companies provide a public official's “personal opinion” and citizens can better
understand their decisions or debate them. Gil-Garcia and Gonzalez-
Miranda (2010) propose several elements that constitute channels
for participation in government websites. Table 2 summarizes these
Local Govt. State elements.
Agencies Government
Federal
Government 2.2.3. Information sharing and opportunities for collaboration
How can citizens collaborate with governments using technology?
Fig. 2. Simplified network of actors potentially interacting, participating, collaborating, The promotion of collaboration can be seen in at least two ways: ei-
and sharing information through an internet government portal. ther citizens collaborate with government or government agencies
S76 R. Sandoval-Almazan, J.R. Gil-Garcia / Government Information Quarterly 29 (2012) S72–S81
Table 1
Social media government applications.
Source: Adapted from (Chun, Shulman, Sandoval-Almazan, & Hovy, 2010).
Technology Examples
collaborate with one another. For citizen collaboration, there are non-government actors. This collaboration synergy could be promot-
many relatively easy and useful options. For example, e-government ed through different web-based platforms — social media, Web 2.0,
portals could have warning systems to detect problems, riots, or secu- collaboration, and e-learning tools (Lips, 2010).
rity threats and citizens can use these tools in order to alert authori-
ties. Dawes and Prefontaine (2003) mention that collaboration rests
on a working philosophy that relationships are evolving and dynamic, 3. Research design and methods
where citizens may want to alert government about public service
failures one day and discuss public policy issues the next. Some citi- As an empirical effort, we collected data from local governments
zens may want to be more involved and collaborate in certain aspects in Mexico. The main objective of this research is to understand to
or topics only and governments should have the flexibility for differ- what extent local government websites are including tools and strat-
ent types of collaborations (Noveck, 2009). egies to foster interaction, participation, collaboration, and informa-
Similarly, there are several forms of collaboration within the tion sharing among government agencies and between government
boundaries of government (Ferro & Sorrentino, 2010; Gil-Garcia et agencies and other social actors such as citizens, businesses, and
al., 2009; Navarrete et al., 2010; Noveck, 2009; Zheng et al., 2009). non-profit organizations. In Mexico there are more than 2400 munic-
One option is collaboration among different municipalities like in ipal governments located in 32 states. This diversity creates some
some shared services initiatives. Another is to collaborate and share challenges for the analysis because most local governments in rural
information among multiple government agencies or with other areas do not use any ICTs. Therefore, two different instruments
levels of government. All these cases present good opportunities for were developed and administered. For the urban local governments,
collaboration using portals and Web 2.0 tools and applications. questions took into consideration their relatively high development
Thus, an e-government portal could become the center for opportuni- and use of information technologies. For the rural municipalities,
ties to collaborate and share information among government and the instrument considered some basic uses of ICTs. There was also a
Table 2
Channels for participation.
Source: (Gil-Garcia & Gonzalez-Miranda, 2010).
Element Description
Offline principles These not only involve education, but also policies and legislation aimed at providing the foundation of
democracy and citizen participation.
Online involvement These could be observed in the form of tools used to promote and motivate citizen participation
through websites.
Information for users As a bare minimum, it includes names of officials, street addresses, electronic addresses, telephone numbers,
and business hours.
Discussion forums These are areas for debate on specific topics that allow for the expression of deliberative comments.
Online contributions Know as weblogs or simply blogs, they are a form of active participation used to increase awareness
about general topics and to record opinions, reports, stories and other types of articles related to a
particular community topic, sector, or territory.
Real time conversations These are tools similar to discussion tables, except that they are held at specific and limited times.
Surveys These can be used to gauge the different perceptions related to implemented policies. The advantage of
surveys is that they collect quantifiable data that is easy to analyze and understand and they require
minimal equipment or skill.
Voting E-voting implies the introduction of technological components at some or all stages of the voting process,
which makes them more difficult to implement than traditional voting methods.
Feedback and results of involvement These are results of consultations and discussions, which must be published afterwards in order to provide
participants with feedback and to strengthen confidence in the decision making process.
R. Sandoval-Almazan, J.R. Gil-Garcia / Government Information Quarterly 29 (2012) S72–S81 S77
One of the objectives of this research is to produce lessons for Variable Findings
the development of a more effective framework to understand local
Information technologies* Fax: 95%
e-government, including tools and applications for interaction, partic- Internet: 100%
ipation, collaboration, and information sharing. The sample included Mobile: 88%
108 municipalities. The local governments included in the sample Number of computers More than 30: 83%
20–30 computers: 9%
were: (1) municipalities that were state capitals, (2) the second larg-
10–20 computers: 5%
est cities in each state, and (3) at least one rural city in each state. All 1–5 computers: 3%
of them had to have an official government website. Operating system* Win XP: 99%
Once the URLs of the websites were validated, the data collection took Win Vista: 72%
place. The research team visited the websites for about 30–50 minutes Win 98: 15%
Win Me: 5%
and filled out an online survey according to the respective questionnaires.
Linux: 17%
For the urban local governments (71 cases), the complete survey of 60 Mac OSx: 10%
questions was applied. In the case of the rural local governments (37 Software* Power Point, Word and Excel: 97%
cases), a partial section of the survey (35 questions) was applied. The Open Source: 26%
Information systems* Data Base: 100%
instrument includes questions related to website format, content infor-
ERP: 13%
mation, security issues, and technologies in use. For the technology- CRM: 6%
related questions, data about hardware and software use was not avail-
*Note: Respondents could choose several options; therefore, the total is more than
able from the local government websites. Phone interviews were used 100%.
to collect the data, but the research team was able to perform only 59 in-
terviews, either because some local governments do not have a valid tele-
phone number or the person responsible for the technical features was
never available. Data collection took place from March to May of 2009. terms of operating systems, almost all local governments (99%) had
Both questionnaires – urban and rural – were tested and piloted on at least one computer with Windows XP and 72% had at least one
three websites prior to their general application and two researchers val- computer with Windows Vista. Not surprisingly, Linux (17%) and
idated the database. Mac OS (10%) are the operating systems that local governments use
the least. In terms of productivity software, it is interesting that al-
4. Analysis and results though most local governments (97%) use Microsoft Office products,
many of them (26%) are also using some open source software.
This section presents our main findings. First, a general overview All local governments currently use a database system for their
of the results is described and analyzed in terms of urban versus operations, but only 13% percent respond that are using an Enterprise
rural local governments. Then we present the results according to Resource Planning (ERP) system and 6% are using a Customer Rela-
each of the main functions of government websites described at the tionships Management (CRM) system. Regarding social media, 77%
beginning of this paper: (1) information display; (2) provision of ser- mentioned that they do not use wikis and blogs. Similarly, 80% of
vices; (3) tools for interaction; (4) channels for participation; and (5) local governments said they did not use Facebook or Hi5. Only 15%
opportunities for collaboration (see Fig. 1). of local government said they would possibly use these kinds of appli-
Within the urban websites evaluated, the city of Mérida in the cations in the future.
southern state of Yucatán was ranked highest, obtaining 189 points
out of a maximum of 240, implying that there is still work to be
done and important opportunities exist for improvement in urban 4.1. Information display
local e-government in Mexico. The lowest ranked was Salina Cruz in
the state of Oaxaca that reached 23 points in this evaluation. In gen- Regarding information display, 27% of the municipalities show
eral, 20 urban local governments were above the mean. The average very limited multimedia information (see Fig. 3). However, 60% do
score of urban local governments was 113 points. In the evaluation present updated information. On the side of information transparen-
of rural local governments, Zacaltelco in the state of Tlaxcala obtained cy, 57% show financial statements and 48% reveal the wages and sal-
the highest score: 140 points out of a maximum of 150 points. The aries of the public officials; 47% of the municipal portals reveal
lowest score was San Francisco Romo in the state of Aguascalientes, information on public purchases and procurement. Finally, only 27%
which reached only 23 points. In general terms, 20 rural local govern- use an electronic bulletin to publicize governmental news.
ments (out of 37) are in a very basic stage of technology use, 14 could
be considered mid-level, and only four are advanced in their use of
ICTs. Not surprisingly, research results show that the variables linked
with information display obtained the highest values. In addition, it
seems clear that rural local governments score significantly lower
than urban local governments in all variables, but more so in technol-
ogy and website format.
The technology component was evaluated through telephone in-
terviews with the individuals responsible for technology or informa-
tion systems in each local government. As mentioned before, we
were able to interview people from 59 local governments. Table 3
shows the main findings from the technology-related questions. All
contacted local governments said that they had internet access, al-
though the number of access points and their quality was not evalu-
ated in this study. About 90% of local governments used mobile
phones and 95% had fax machines. Most local governments (83%)
had more than 30 computers and only 3% had fewer than five. In Fig. 3. Information display variables by percentage.
S78 R. Sandoval-Almazan, J.R. Gil-Garcia / Government Information Quarterly 29 (2012) S72–S81
4.2. Provision of services There were no tools or applications for collaboration on any of the
websites. We were looking for wikis or specific applications for col-
Forty percent (40%) of the e-government municipal sites present laboration among government agencies or between government
links to download forms for services; 60% present an online service agencies and other social actors, but none were found.
catalog; and 28% have a link to search for employment (see Fig. 4). Finally, many local governments are below average in each of the
But only 9% offer the capability to pay for services online and 34% components, which shows that there are a few local governments
offer different forms of payment like bank transfers. with very high scores, while most of them obtained low scores in all
components. This disparity is also an indicator of the heterogeneity
of local governments in Mexico in terms of resources and capabilities.
4.3. Tools for interaction Not surprisingly, the components with the highest scores are the ones
related to information display and service provision and the compo-
The average grade for interaction was 49 out of a possible maxi- nents with the lowest scores are the ones related to interaction, par-
mum of 88 (see Fig. 5). One way to interact is by providing the ticipation, and collaboration (see Fig. 7).
email directory for public officials; only half (51%) of the municipali-
ties had this tool. Only 29% percent of the websites provided contact 5. Discussion and implications
information for the Webmaster of the municipal site. Advanced
tools like blogs were present in only 17% of the municipalities and The main practical goal of this kind of research is to provide gov-
19% provided RSS access on their website. Only 15% of the municipal- ernment officials with valuable information for the development of
ities included a specific interaction tool on their site, such as a chat policies that promote the use of technology in local governments, es-
feature, forum, or virtual assistant. pecially in terms of participation and collaboration. Mapping the
weaknesses and strengths of the impact of ICTs could help leaders
to make decisions regarding public policies and reduce the gap
4.4. Channels for participation among local governments in terms of technology. It could also be use-
ful as a roadmap for building a local governance model based on com-
Despite the importance of citizen participation, there are a limited munication, engagement, and information sharing through tools and
number of interaction channels on Mexican local government web- applications included in government websites. Another application
sites. Only 22% of the municipalities had surveys about political topics could be the assessment of open government strategies. The use of
or public policy on their portals. No other channels for participation ICTs could help mitigate some limitations, but it is still difficult for
were found (see Fig. 6).
Fig. 5. Tools for interaction variables by percentage. Fig. 7. All government website functions by average percentage.
R. Sandoval-Almazan, J.R. Gil-Garcia / Government Information Quarterly 29 (2012) S72–S81 S79
6th International Conference, EGOV 2007 from http://www.springerlink.com/ Gould, E., Gomez, R., & Camacho, K. (2010). Information Needs in Developing Countries:
content/9x334157g700u405/?p=a61c45c139434bb2801de27c80c11d10&pi=34 How Are They Being Served by Public Access Venues? Paper presented at the AMCIS
Baker, P. M. A., Hanson, J., & Myhill, W. N. (2009). The promise of municipal WiFi and from http://aisel.aisnet.org/81625amcis2010/9.
failed policies of inclusion: The disability divide. Information Polity, 14(1–2), 47–59. Gouscos, D., Mentzas, G., & Georgiadis, P. (2001). PASSPORT, a novel architectural
Ballejos, L. C., & Montagna, J. M. (2010). Identifying interorganisational network; a model for the provision of seamless cross-border e-government services. Paper
factor-based approach. International Journal of Networking and Virtual Organisation, presented at the Proceedings of the 12th International Workshop on Database and Ex-
7(1), 1–22. pert Systems Applications.
Barnes, S. J., & Vidgen, R. (2007). Interactive E-government: Evaluating the web site of Hahamis, P., Iles, J., & Healy, M. (2005). e-Government in Greece: Bridging the gap be-
the UK inland revenue. International Journal of Electronic Government Research, 3 tween need and reality. Electronic Journal of e-Government, 3(4), 185–192.
(1). Inglehart, R. (1997). Postmaterialist values and the erosion of institutional authority. In
Bochicchio, M. A., Celentano, M. G., & Longo, A. (2004). Innovation needs in the e- J. S. Nye, P. D. Zelikow, & D. C. King (Eds.), Why people don't trust government
government scenario: A survey. In R. Traunmüller (Ed.), Electronic government, (pp. 217–237). : Harvard College.
Vol. 3183. (pp. 347–354)Heidelberg: Springer Berlin. Kaylor, C., Deshazo, R., & Van Eck, D. (2001). Gauging e-government: A report on
Boyer-Wright, K. M., & Kottemann, J. E. (2008). High-level factors affecting global avail- implementing services among American cities. Government Information Quarterly,
ability of online government services. Paper presented at the Proceedings of the 41st 18(4), 293–307.
Annual Hawaii International Conference on System Sciences (HICSS '08). Klievink, B., & Janssen, M. (2010, 5–8 January 2010). Simulation games for collaborative
Buccoliero, L., & Bellio, E. (2010). Citizens web empowerment in European municipal- development in e-Government. Paper presented at the 43rd Hawaii International
ities. Journal of E-Governance, 33(4), 225–236. Conference on System Sciences (HICSS-43), Koloa, Kauai, HI.
Canestraro, D. S., Pardo, T. A., Raup-Kounovsky, A. N., & Taratus, D. (2009). Regional Kun Chang, L., Melih, K., Sangjae, L., & Gyoo Gun, L. (2008). User evaluations of tax filing
telecommunication incident coordination: Sharing information for rapid response. web sites. Online Information Review, 32(6), 842.
Information Polity, 14(1–2), 113–126. Kunstelj, M., & Decman, M. (2005). Current state of e-government in Slovenian munic-
Cassell, M., & Hoornbeek, J. (2010). Engaging citizens on the internet: An assessment of ipalities. Electronic Journal of e-Government, 3(3), 117–128.
local governments in Ohio. International Journal of Electronic Government Research, Layne, K., & Lee, J. W. (2001). Developing fully functional E-government: A four stage
6(2), 68–85. model. Government Information Quarterly, 18(2), 122–136.
Chatzopoulos, K. -C., & Economides, A. A. (2009). A holistic evaluation of Greek munic- Lee, J., & Rao, H. R. (2007). Exploring the causes and effects of inter-agency information
ipalities' websites. Electronic Government, an International Journal, 6(2), 193–212. sharing systems adoption in the anti/counter-terrorism and disaster management
Chun, S. A., Shulman, S., Sandoval-Almazan, R., & Hovy, E. (2010). Government 2.0: domains. Paper presented at the Proceedings of the 8th annual international confer-
Making connections between citizens, data and government. Information Polity, ence on Digital government research: Bridging disciplines & domains.
15(1–2), 1–9. Leenes, R., & Svensson, J. (2002). Size matters — Electronic service delivery by munici-
Clarkson, G., Jacobsen, T. E., & Batcheller, A. L. (2007). Information asymmetry and in- palities? Paper presented at the Electronic government: First international conference,
formation sharing. Government Information Quarterly, 24(4), 827–839. EGOV 2002, Aix-en-Provence, France, September 2–6, 2002: Proceedings from http://
Connolly, R., Bannister, F., & Kearney, A. (2010). Government website service quality: A www.springerlink.com/openurl.asp?genre=article&id=FHNPUFW0Y190LL5M
study of the Irish revenue online service. European Journal of Information Systems, Lips, M. (2010). Rethinking citizen — Government relationships in the age of digital
19(6), 649. identity: Insights from research. Information Polity, 15(4), 273–289.
Dawes, S. S. (1996). Interagency information sharing: Expected benefits, manageable Maldonado, E. A., Maitland, C. F., & Tapia, A. H. (2010). Collaborative systems develop-
risks. Journal of Policy Analysis and Management, 15(2), 377–394. ment in disaster relief: The impact of multi-level governance. Information Systems
Dawes, S. S. (2010, 5–8 January 2010). Information policy meta-principles: Steward- Frontiers, 12(1), 9–27.
ship and usefulness. Paper presented at the 43rd Hawaii International Conference Matheus, R., & Ribeiro, M. M. (2009). Models for citizen engagement in Latin American:
on System Sciences (HICSS-43), Koloa, Kauai, HI. Case studies of public digital budgeting. Paper presented at the Proceedings of the
Dawes, S. S., Cresswell, A. M., & Pardo, T. A. (2009). From “need to know” to “need to 3rd international conference on Theory and practice of electronic governance from
share”: Tangled problems, information boundaries, and the building of public sec- http://0-portal.acm.org.millenium.itesm.mx/citation.cfm?id=1693042.
tor knowledge networks. Public Administration Review, 69(3), 392–402. 1693065&coll=DL&dl=GUIDE&CFID=9763452&CFTOKEN=78811908
Dawes, S. S., & Prefontaine, L. (2003). Understanding new models of collaboration for Matheus, R., Ribeiro, M. M., Vaz, J. C., & Souza, C. A. d. (2010). Case studies of digital par-
delivering government services. Association for Computing Machinery. Communica- ticipatory budgeting in Latin America: Models for citizen engagement. Paper pre-
tions of the ACM, 46(1), 40. sented at the Proceedings of the 4th International Conference on Theory and Practice
Deloitte, R. (2000). Public sector institute at the dawn of E-government: The citizen as a of Electronic Governance.
costumer. Melitski, J., Holzer, M., Kim, S. -T., Kim, C. -G., & Rho, S. -Y. (2005). Digital government
Dobrev, B., Stoewer, M., Makris, L., & Getsova, E. (2002). E-MuniS – electronic munici- worldwide: An e-government assessment of municipal web sites. International
pal information services – best practice transfer and improvement project: Project Journal of Electronic Government Research, 1(1), 1–19.
approach and intermediary results. Paper presented at the Proceedings of the First In- Moon, M. J. (2002). The evolution of e-government among municipalities: Rhetoric or
ternational Conference on Electronic Government. reality? Public Administration Review, 62(4), 424–433.
Ebbers, W. E., Pieterson, W. J., & Noordman, H. N. (2008). Electronic government: Re- Navarrete, C., Gil-Garcia, J. R., Mellouli, S., Pardo, T. A., & Scholl, J. (2010, 5–8 January
thinking channel management strategies. Government Information Quarterly, 25 2010). Multinational e-government collaboration, information sharing, and inter-
(2), 181–201. operability: An integrative model. Paper presented at the 43rd Hawaii International
Evans-Cowley, J. S. (2005). The accessibility of municipal government websites. Journal Conference on System Sciences (HICSS-43), Koloa, Kauai, HI.
of E-Government, 2(2), 75–90. Navarrete, C. A., Mellouli, S., Pardo, T., & Gil-Garcia, J. R. (2009). Information sharing at na-
Ferro, E., & Sorrentino, M. (2010). Can intermunicipal collaboration help the diffusion tional borders: Extending the utility of border theory. Paper presented at the Proceed-
of E-Government in peripheral areas? Evidence from Italy. Government Information ings of the 42nd Annual Hawaii International Conference on System Sciences (HICSS '09).
Quarterly, 27(1), 17–25. Noveck, B. S. (2009). Wiki government: how technology can make government better, de-
Fink, D. (2010). Road safety 2.0 — insights and implications for government. Paper pre- mocracy stronger, and citizens more powerful. : Brookings Institution Press.
sented at the BLED from http://aisel.aisnet.org/bled2010/22 Ntaliani, M., Costopoulou, C., Manouselis, N., & Karetsos, S. (2009). M-government ser-
Gant, J. P. (2003). Delivering e-government services through the Access Indiana Informa- vices for rural SMEs. International Journal of Electronic Security and Digital Forensic,
tion Network. Albany, NY: Center for Technology in Government. 2(4), 407–423.
Gant, J. P., & Gant, D. B. (2002). Web portal functionality and state government e- Ong, C. -S., & Wang, S. -W. (2009). Managing citizen-initiated email contacts. Govern-
service. Paper presented at the System Sciences, 2002. HICSS. Proceedings of the ment Information Quarterly, 26(3), 498–504.
35th Annual Hawaii International Conference on from http://ieeexplore.ieee.org/ Papenfuß, U., & Schaefer, C. (2010). Improving public accountability by aligning report-
xpl/freeabs_all.jsp?arnumber=994073&abstractAccess=no&userType=inst ing to organizational changes in public service provision — An empirical Internet
Gelders, D., Brans, M., Maesschalck, J., & Colsoul, N. (2010). Systematic evaluation of study of all Austrian, German and Swiss towns and states from an agency-theory
public participation projects: Analytical framework and application based on two perspective. International Review of Administrative Sciences, 76(3).
Belgian neighborhood watch projects. Government Information Quarterly, 27(2), Pardo, T. A., Gil-Garcia, J. R., & Burke, G. B. (2008). Governance structures in cross-
134–140. boundary information sharing: Lessons from state and local criminal justice initia-
Gil-Garcia, J. R., Chun, S. A., & Janssen, M. (2009). Government information sharing and tives. Paper presented at the Proceedings of the 41st Annual Hawaii International Con-
integration: Combining the social and the technical. Information Polity, 14(1–2), ference on System Sciences (HICSS '08).
1–10. Parent, M., Vandebeek, C. A., & Gemino, A. C. (2005). Building citizen trust through e-
Gil-Garcia, J. R., & Gonzalez-Miranda, F. (2010). E-government and opportunities for government. Government Information Quarterly, 22(4), 720–736.
participation: The case of the Mexican state web portals. In C. Reddick (Ed.), Citi- Paris, M. (2006). Local e-government and devolution: Electronic service delivery in
zens and e-government: Evaluating policy and management (pp. 56–74). Hershey, Northern Ireland. Local Government Studies, 32(1).
PA: IGI Global. Reddick, C. G. (2004). A two-stage model of e-government growth: Theories and em-
Gil-Garcia, J. R., & Luna-Reyes, L. F. (2006). Integrating conceptual approaches to e- pirical evidence for U.S. cities. Government Information Quarterly, 21(1), 51–64.
government. In M. Khosrow-Pour (Ed.), Encyclopedia of e-commerce, e- Resca, A. (2010). A question of progress: Face-to-face relationships versus face-to-
government and mobile commerce (pp. 636–643). Hershey, PA: Idea Group Inc.. screen relationships. Paper presented at the MCIS 2010 from http://aisel.aisnet.org/
Gil-Garcia, J. R., & Martinez-Moyano, I. J. (2007). Understanding the evolution of e- mcis2010/71
government: The influence of systems of rules on public sector dynamics. Govern- Rodríguez, R. A., Welicki, L., Giulianelli, D. A., & Vera, P. M. (2008). Measurement frame-
ment Information Quarterly, 24(2), 266–290. work for evaluating e-governance on municipalities websites. Paper presented at
Gottschalk, P. (2009). Maturity levels for interoperability in digital government. Gov- the Proceedings of the 2nd International Conference on Theory and Practice of Elec-
ernment Information Quarterly, 26(1), 75–81. tronic Governance.
R. Sandoval-Almazan, J.R. Gil-Garcia / Government Information Quarterly 29 (2012) S72–S81 S81
Sandoval-Almazan, R., & Castaneda, M. A. (2010). Gobierno electrónico en los munici- public safety: Cartographic insights on rational choice and institutional explana-
pios: el caso de Toluca. Paper presented at the Proceedings of the 11th Annual Inter- tions. Information Polity, 14(1–2), 13–29.
national Digital Government Research Conference on Public Administration Online: Zheng, L., Yang, T. -M., Pardo, T., & Jiang, Y. (2009). Understanding the “boundary” in
Challenges and Opportunities. information sharing and integration. Paper presented at the Proceedings of the
Sandoval-Almazan, R., Diaz-Murillo, G., Gil-Garcia, J. R., & Luna-Reyes, L. F. (2010). Web 42nd Annual Hawaii International Conference on System Sciences (HICSS '09).
2.0 en los portales estatales en México: una primera aproximación. Revista de
Administración Pública, XLV(121).
Sandoval-Almazan, R., & Gil-Garcia, J. R. (2006). E-government portals in Mexico. In M.
Khosrow-Pour (Ed.), (1st ed.). Encyclopedia of e-commerce, e-government and mo- Rodrigo Sandoval-Almazán is an Assistant Professor in the Research Center of the Busi-
bile commerce, Vol. II. (pp. 367–372)Hershey PA California: Idea Group. ness College at the Autonomous State University of Mexico in Toluca City and a Research
Sandoval-Almazan, R., & Gil-Garcia, J. R. (2008). E-government portals in Mexico. Elec- Fellow at the Center for Development of Information Technologies and Electronics of the
tronic Government: Concepts, Methologies, Tools and Aplications, 1(1), 1726–1734. ITESM. He has lectured on topics such as information systems for business, information
Sandoval-Almazan, R., & Gil-Garcia, J. R. (2008). Limitations of evolutionary approaches systems strategy for business, electronic commerce development, digital divide in emer-
to e-government. In G. D. Garson, & M. Khosrow-Pour (Eds.), Handbook of research gent countries, organization theory, database applications, Statistics, web development,
on public information technology. Hershey, PA: IGI Global. quantitative analysis and modeling, research methods, public administration theory,
Sandoval-Almazan, R., & Gil-Garcia, J. R. (2009). A preliminary assessment of a state legisla- and local government management. Dr. Sandoval-Almazán is the author or co-author of
ture web site in Mexico. Paper presented at the Proceedings of ongoing research, general articles in the Handbook of Research on Public Information Technology; Journal of Informa-
development issues and projects of EGOV 09 8th International Conference, Linz, Austria. tion Technology for Development; Electronic Journal of Information Systems Research; and
Sandoval-Almazan, R., & Gil-Garcia, J. R. (2009, August-September 2009). A prelimi- Espacios Públicos. His research interests include electronic government, information tech-
nary assessment of a state legislative. Paper presented at the Eight International nologies and organizations, social networks, digital divide technology, and multi-method
EGOV Conference, Linz, Austria. research approaches. Dr. Sandoval-Almazán has a Bachelors Degree in Political Science
Sandoval-Almazan, R., & Gil-Garcia, J. R. (2010). Assessing local e-government: An ini- and Public Administration, a Masters in Management focused on Marketing, and a Ph.D.
tial exploration of the case of Mexico. Paper presented at the International Confer- in Management with Information Systems.
ence on Theory and Practice of Electronic Governance.
Susan, X. (2004). Web usage statistics and Web site evaluation: A case study of a gov-
ernment publications library Web site. Online Information Review, 28(3), 180. J. Ramon Gil-Garcia is an Associate Professor in the Department of Public Administration
Tapscott, D., & Wiliams, A. (2010). Macrowikinomics. : Portfolio Hardcover. and the Director of the Data Center for Applied Research in Social Sciences at Centro de
Tolbert, C. J., & Mossberger, K. (2006). The effects of e-government on trust and confi- Investigación y Docencia Económicas (CIDE) in Mexico City. Currently he is also a Re-
dence in government. Public Administration Review, 66(3), 354–369. search Fellow at the Center for Technology in Government, University at Albany, State
White, G. B., & DiCenso, D. J. (2005). Information sharing needs for national security. University of New York (SUNY) and a Faculty Affiliate at the National Center for Digital
Paper presented at the Proceedings of the 38th Annual Hawaii International Confer- Government, University of Massachusetts Amherst. He is a former Fulbright Scholar. Dr.
ence on System Sciences (HICSS'05) — Track 5 from http://www.computer.org/ Gil-Garcia is the author or co-author of articles in Government Information Quarterly, Jour-
portal/web/csdl/doi/10.1109/HICSS.2005.320 nal of the American Society for Information Science and Technology, European Journal of Infor-
Wigand, F. D. L. (2010, 12–14 April 2010). Twitter in government: Building relation- mation Systems, The International Public Management Journal, Journal of Government
ships one tweet at a time. Paper presented at the Information Technology: New Gen- Information, International Journal of Electronic Government Research, Public Finance and
erations (ITNG), 2010 Seventh International Conference on. Management, International Journal of Cases on Electronic Commerce, and International Jour-
Williams, M. D. (2008). E-government adoption in Europe at regional level. Transform- nal of Electronic Governance, among other academic journals. His research interests include
ing Government: People, Process and Policy, 2(1), 47–59. collaborative electronic government, inter-organizational information integration, digital
Williams, C. B., Dias, M., Fedorowicz, J., Jacobson, D., Vilvovsky, S., Sawyer, S., et al. divide policies, adoption and implementation of emergent technologies, education policy,
(2009). The formation of inter-organizational information sharing networks in public policy evaluation, and multi-method research approaches.