Tourism Management 32 (2011) 1463e1473

Contents lists available at ScienceDirect

Tourism Management
j o u r n a l h o m e p a g e : w w w . e l se v i e r . c o m / l o c a t e / t o u r m a n

Case Study

A strategic website evaluation of online travel agencies

Wen-Chih Chiou a, Chin-Chao Lin b, *, Chyuan Perng c
a Department of Business Administration, National Chin-Yi University of Technology, 35, Lane 215, Chung-San Rd Sec 1, Taiping City, Taichung County 411, Taiwan, ROC
b
Department of Marketing and Distribution Management, Hsiuping Institute of Technology, 11, Gongye Rd, Dali City, Taichung County 412, Taiwan, ROC
c
Department of Industrial Engineering and Enterprise Information, Tunghai University 181 Sec 3, Taichung Harbor Rd., Taichung 407, Taiwan, ROC

article info abstract

Article history: Online travel Web sites have been the most frequently visited online information facilities by travelers. To evaluate the
Received 23 May 2010 effectiveness of a travel Web site, the Web site manager should regularly check whether or not it is fulfilling the objectives that
Accepted 15 December 2010 were established for it. This research uses a strategic Web site evaluation framework to introduce a five-stage process for
examining the consistency of Web site’s presence and its intended strategies. Two leading online travel agencies with different
Keywords: business strate-gies are selected to demonstrate methods of implementing a strategic evaluation framework and to compare the
Website evaluation
evaluation results. A hierarchical evaluation structure is introduced to explicitly delineate the two Web sites’ different strategy
Online travel agencies
intentions and related evaluation criteria. Results show that an individual Web site’s strategy-inconsistent criteria can be easily
Strategy consistency
identified through a gap analysis and criteria performance matrix. A strategy-inconsistent dimension can be discovered through
a radar chart analysis of the 4PsC (Product, Promotion, Price, Place, and Customer Relationship) dimensions and a transaction
phases analysis.

2010 Elsevier Ltd. All rights reserved.

1. Introduction There are only a few successful studies on what motivates users to browse
and make purchases on travel Web sites (Law et al., 2010). Several review
Along with the rapidly increasing popularity of the Internet, travel Web studies have noted that there is no univer-sally accepted technique or standard
sites have become some of the most frequently visited online information for Web site evaluation (Law et al., 2010; Morrison, Taylor; & Douglas, 2004;
facilities by travel planners (Choi, Lehto, & Oleary, 2007; Law & Leung, 2000; Tsai, Chou, & Lai, 2010). Although the findings of these studies have revealed
Zhou & DeSantis, 2005). The Internet is leading businesses into a new era in the most crucial and prevalent features of successful tourism Web sites in
the field of communication and is changing business transactions. The market general, these identified features may not necessarily be appli-cable to every
of products and services in the tourism industry relies heavily on information Web site because organizations have their own Web site development strategies
and has a highly segmented structure (Roney & Özturan, 2006; Thorn & Chen, to attain their goals and objectives. Web sites are developed based on
2005). In fact, the tourism industry is one of the world’s largest industries organizational goals and objec-tives. Therefore, they should be reviewed
adopting the Internet as the medium for e-business revolution. Moreover, the regularly to determine whether they are fulfilling the reasons for which they
Web is now the most widely used tool in conducting research on tourist were developed, and it is not necessary to have a universal standard in assessing
informa-tion and promoting regional tourism; it is also cheaper compared with the success or effectiveness of a Web site because each Web site is designed
other forms of promotion and advertising (Horng & Tsai, 2010; Standing & for a specific reason (Clyde, 2000).
Vasudavan, 2000; Stepchenkova, Tang, Jang, Kirilenko,

& Morrison, 2010). Maintaining an effective Web site is thus vital for a Many previous studies have adopted user surveys in investi-gating user
business-to strengthen its customer relationships and enlarge its market perceptions of selected Web sites, whereas only a few studies have adopted
segment (Law, Qi, & Buhalis, 2010). expert-based evaluations. User-based surveys can be regarded as an external
evaluation method to examine whether a Web site is “doing the thing right” in
meeting user expectations. Nevertheless, to examine whether a Web site is
“doing the right thing” in meeting its Web strategy requirements, an internal
* Corresponding author. Tel.: þ8864 24961100; fax: þ8864 24961187.
evaluation should be conducted by a panel of experts before the conduct of
E-mail addresses: chiouwc@ncut.edu.tw (W.-C. Chiou), cclkevin@gmail.com, external surveys. The internal evaluation should persist until the Web site
kevinlin@mail.hit.edu.tw (C.-C. Lin), perngc@thu.edu.tw (C. Perng). becomes consistent with its

0261-5177/$ e see front matter 2010 Elsevier Ltd. All rights reserved.
doi:10.1016/j.tourman.2010.12.007
1464 W.-C. Chiou et al. / Tourism Management 32 (2011) 1463e1473

strategy, after which an external evaluation may follow. Thus, this study aims After reviewing numerous evaluation frameworks introduced within the
to use a strategic Web site evaluation framework to develop a systemic last decade, Chiou, Lin, and Perng (2010) identified three issues that should be
evaluation process in examining the consis-tency of Web site presence and further addressed. First, an evaluation framework must be process oriented to
intended strategy. The following section reviews recent studies on Web site identify crucial activities in each transactional phase. For instance, Roney and
evaluation. Section 3 introduces our proposed strategic evaluation Özturan (2006) proposed a process oriented framework for evaluating Turkish
methodology. In Section 4, the methodology is applied to two leading travel travel agencies. The framework consists of three levels of func-tionalities:
Web sites in Taiwan. The last section includes a brief summary of this study, corporate information, before-sale information, and sales and after-sales
some managerial implications, and suggestions for evalu-ating travel Web activities. Web site functionalities are analyzed through a business process
sites. perspective; hence, researchers and practitioners can effectively identify key
customer-related activi-ties. Second, a hybrid approach that considers the role
of IS as a support factor in marketing, instead of a combined approach, suggests
2. Study background that IS factors should be embedded into marketing factors as facilitators of e-
commerce. By doing so, the confusion in the classification of criteria can be
Various approaches in evaluating Web sites have been proposed by eliminated. Third, existing studies have proposed various frameworks with
academic researchers since the late 1990s. The most common approaches extensive factors and criteria in evaluating Web sites. Unfortunately, none of
include content analysis, benchmarking, survey, experiment, case study, and these frameworks addresses the issue of the relationship between Web strategy
automatic evaluation. In particular, content analysis and benchmarking have and evaluation factors/criteria. A framework that considers strategy ensures
been used widely by tourism and hospitality field researchers. For instance, that Web site presence is consistent with its pre-defined goals and objectives.
Law et al. (2010) reviewed 75 papers, 27 of which adopted content analysis and
10 used benchmarking. These studies are generally divided into two broad
categories: quantitative and qualitative. Many researchers are presently
integrating quantita-tive and qualitative methods in their studies. These studies In response to these three issues, Chiou et al. (2010) proposed a strategic
are further generally divided into five evaluation approaches: (i) counting, (ii) evaluation framework (Fig. 1). The framework is con-structed based on the
user judgment, (iii) automated, (iv) numerical computation, and (v) no actual goals and objectives of each Web site. As mentioned earlier, the framework
evaluation. Law et al. (2010) also concluded that the adoption of a combination must be transactional process oriented. It should include information,
of methods provides a range of results that can satisfy the different needs of the agreement, and settlement phases. The information phase starts when
entire range of stakeholders. Hence, what existing studies seem to have in prospective buyers enter the e-commerce system and lasts until they decide to
common is a general agreement that assessing the effectiveness or performance place an order or leave the system. The agreement phase involves negotia-tions
of a Web site requires a multi-dimensionaldrather than a between prospective buyers and sellers, which are finalized by contracts.
unidimensionaldapproach or measure (Park & Gretzel, 2007). Eventually, the contracts are executed in the settle-ment phase according to
stipulated conditions; product delivery and after-sales interactions take place
during this phase. Chiou et al. (2010) collected representative criteria from 83
papers published in prestigious journals as the initial pool. To identify the most
Chiou, Perng, Tsai, and Lin (2008) selected 139 articles from 21 leading frequently used factors, the criteria were further classified into 12 unified factors
journals to identify trends in Web site evaluation and analyze frameworks and as suggested by Park and Gretzel (2007). The top three factors are information
criteria proposed by different researchers. They classified these papers into quality, ease of use, and responsiveness.
three major categories: (i) infor-mation system (IS), (ii) marketing, and (iii)
combination. In an IS-oriented study, 75% of the evaluation factors are
technology related. On the other hand, in a marketing-oriented study, over 75%
of the valuation factors, such as advertising, promotion, online trans-action, After eliminating repetitive items, merging similar items, and condensing
order confirmation, and customer service, are marketing related. Combination- sub-attributes to higher-level criteria, Chiou et al. (2010) kept 53 criteria in the
T
oriented studies feature a mixture of IS and marketing factors, and they have criteria pool (Table 1). Criteria in Table 1 bearing the superscript “ ” are
become prominent since the burst of the dot-com bubble. supported by information technology. This is an example of a hybrid concept.
These criteria are categorized into five factors: product, promotion, price, place,

Fig. 1. A strategic Web site evaluation framework.

 W.-C. Chiou et al. / Tourism Management 32 (2011) 1463e1473 1465

Table 1
Criteria pool for Web site evaluation.

Factors/Criteria Number of supported studies Factors/Criteria Number of

supported studies
Place Product
Ease of navigationT 49 Product details 28
Content relevancy and usefulness 44 Product comparisonT 13
Appealing and consistent styleT 44 Product search or assortmentT 10
Logical structureT 39 Product variety 10
Security protectionT 38 Hierarchical product category 7
Ease of online transactionT 35 Product quality 4
User-friendly interfaceT 34 Price
Comprehensive content coverage 33 Price details 14
Loading and processing speedT 32 Competitive price 5
Up-to-date content 31 All relevant charges details 5
Proper multimediaT 30 Price comparisonT 4
Well and quick linkageT 29 Promotion
Searching mechanismT 26 Promotion campaign 17
Easy to understand and read 27 Reputation and credibility of the site 15
Ease of accessT 25 Company and brand recognition 13
Reliable and innovative systemT 24 Purchasing guarantee 10
Accuracy 24 Advertising and banner 7
Easy to find target informationT 22 Customer Relationship
Online assistance and helpT 16 Interactive communicationsT 37
Data retrieve mechanismT 14 Customized serviceT 28
Playfulness 13 Privacy policy 25
Convenient payment methodsT 12 Quick response to customerT 25
Know the present locationT 10 Customer service supportT 23
Overview of selected itemsT 6 Member communityT 19
Easy to cancel or modify orderT 5 Order status inquiry and trackingT 17
Valuable bundles or product suggestion 13
Delivery product as promised 10
Customized offeringsT 9
Convenient delivery options 8
Ease of registrationT 7
Easy to return product 3

Source: Chiou et al. (2010)

T
Represents the hybrid criterion supported by information technology.

and customer relationship (4PsC). These factors are used later for data analysis. selected to express the semantic decision-making process of eval-uators. The
The 4PsC factors are expressed in different shades to represent different rating of each criterion is determined by the group decision of a panel of
relative impacts in three phases. The evaluation criteria are selected from a experts. Lastly, a performance matrix chart is introduced to identify strategy-
criteria pool in response to the goals and objectives of Web sites. inconsistent criteria. The following section explains the steps involved in each
stage of the evaluation process.
The strategic evaluation framework provides managers an internal
evaluation mechanism to examine whether a Web site is consistent with its goal
and objectives. The proposed framework is different from the frameworks of Stage one: Identification of Web site strategy and criteria. Step 1:
existing studies in two aspects. First, most existing frameworks are generally Identifying the goals and objectives of Websites. A personal in-
applicable to measurements of Web site usability, accessibility, design, quality, depth interview with managers is suggested. A goal is a broad vision
content, user satisfaction, user acceptance, and loyalty. The stra-tegic of a site that provides a general description of itself, such as “travel site
framework adopts the goals and objectives of a Web site as guidelines in with the best service quality.” Objectives, on the other hand, are those
selecting relevant criteria to evaluate how well a Web site strategy has been that accomplish the goal of the Web site, such as “providing the most
accomplished. Second, most existing studies focus on the attitude and behavior competitive prices.”
of users toward the design and content of Web sites. The strategic framework
uses the manager’s viewpoint in examining the gap between “what the manager Step 2: Selecting relevant criteria with regard to an
wants” and “what a Web site is” according to experts. objective. For instance, a travel Web site sets objective i (noted as Oi)
as “providing customers with a variety of tour package selections,” and
the related activity is “aligning with other business organizations to
3. Methodology of the five-stage evaluation process introduce special tours, such as firework festivals or mountain biking.”
A related criterion j (noted as Cij), such as “valuable bundles or product
Evans and King (1999) suggested that any assessment tool has five sugges-tions” (Table 1), is selected as a pertinent criterion. In an effort
components: categories (broad areas to be investigated), factors (specific to confirm that a criterion is fully representative of an objective, group
elements comprising each category), weights (importance placed on factors), selection is performed to attain consensus between the researcher and
ratings (scores assigned to each factor), and weighted scores (an overall the manager.
compilation based on both weights and ratings). Based on this concept, a
hierarchical structure of Web site strategy is introduced to delineate the Step 3: Constructing a hierarchical evaluation structure. The
relationship between each category (strategic objectives) and relevant factors structure is designed to delineate the relationship between the goals and
(criteria). In determining criteria weights, fuzzy linguistic terms are objectives of Web sites and their related criteria. A hierarchical
evaluation structure is helpful in cause-and-effect analyses.
1466 W.-C. Chiou et al. / Tourism Management 32 (2011) 1463e1473

Step 4: Assigning weights to each criterion. Managers thoroughly approach is used to assess the consistency of a Web site strategy. To
understand the goals and objectives of their Web sites; hence, they are make a proper group decision in rating the criteria, Robbins (1994)
asked to rate the importance of each criterion. Seven linguistic terms suggested that there should be five to seven experts.
are used: “very unimpor-tant,” “unimportant,” “somewhat unimportant,”
“neutral,” “somewhat important,” “important,” and “very important”. Step 2: Rating each criterion. After identifying themselves on the
Linguistic terms, instead of the Likert scale, are used because the Web-based instrument, evaluators rate each criterion using a linguistic
assignment of criterion importance involves the uncer-tainty and term; they express their agreement or disagreement with statements
fuzziness of human decision-making. Zadeh (1995) suggested that the under each criterion. The linguistic terms are “strongly disagree,”
fuzzy theory is more pertinent in delineating fuzzy characteristics while “disagree,” “some-what disagree,” “neutral,” “somewhat agree,” “agree,”
people are making judgments. In fact, fuzzy data may be expressed in and “strongly agree”. As mentioned earlier, these fuzzy linguistic terms
linguistic terms or in fuzzy numbers. In transforming linguistic terms can be transformed into one of the following crisp scores (Sijk, where i
into crisp numbers, Chen and Hwang (1992) proposed a simple and
is an objective, j is its related criteria, and k is an evaluator): 0.09, 0.23,
effective methodology to solve fuzzy multiple attributes of the decision- 0.36, 0.50, 0.64, 0.78, and 0.91.
making problem. According to the conversion scales of fuzzy numbers,
these fuzzy linguistic terms can be transformed into one of the Stage four: Criteria weights and score calculation.
following crisp weights (Wij, where i is an objective and j is its related Step 1: Normalizing criteria weights. Criteria weights are normalized
criteria): 0.09, 0.23, 0.36, 0.50, 0.64, 0.78, and 0.91. to compare conveniently the relative importance of criteria under each
objective. The normalized weight (NWij) is calculated as follows:

Stage two: Web-based evaluation instrument development. Step 1: NW W

ij
ij (1)
Transforming criteria into questions. The listed criteria are n

conceptual and general. Therefore, researchers must transform them ¼P W

j¼1 ij
into specific and practical questions that fit the Web site and the where Wij is the weight of criterion j with respect to an objective i, and n is
corresponding industry. the number of criteria j under an objective i.
Step 2: Developing a Web-based questionnaire instru-ment. The
online instrument includes two sections. Section 1 includes a summary Step 2: Calculating average scores, weighted scores, and objective
of the interview with the Web site manager, and the hierarchical scores. We need to aggregate individual judgments in a group into a single
structure of the goal, objec-tives, and evaluation criteria of the Web site. representative judgment. According to Saaty (1980), the geometric
This section aims to provide a background of the Web site and a meandnot the frequently used arithmetic meandaccurately represents the
reference for evaluators. Section II is designed for rating scores from consensus of experts, and it is the most widely used in practical
the questionnaires. To improve the quality of the results and save time, applications. In this sense, the geometric mean is used to aggregate the
an evaluation-supporting tool is provided at the end of each question. judgments of a group of n evaluators. The average score of a criterion (ASij)
Human-based evaluation has been proven to be time consuming and is computed as follows:
inconsistent in testing certain features (Law et al., 2010). To address
this problem, two types of evaluation-supporting tools are introduced:
hyperlink and finding. A hyperlink is designed to direct eval-uators to a
vffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi
u
n
relevant Web page. On the other hand, a finding is designed to present ASij un Y S (2)
ijk
results on a specific question. For instance, a hyperlink is provided for ¼ tk 1
¼
the question “Does this Web site list functions in a logical order?” When
an evaluator clicks on this hyperlink, a Web site home page opens so where n is the number of evaluators.
that the evaluator need not open a browser and look for the relevant The weighted score of criterion j (WSij) and the weighted score of an
Web page. An example of finding is the question “Does this Web site objective (OWSi) are calculated using the following equations:
respond to customer questions effec-tively?” To answer this question,
the authors pose the question “How is an order cancelled online?” on the WSij ¼ ASij NWij (3)
customer service page of the Web site and record the response time.
n
When an evaluator clicks on the finding icon, a results page shows the X
posted time, respondent time, and respondent answer to the question. OWSi ¼ WSij (4)
j¼1
Once online question-naires and the evaluation-supporting tool are
ready, a pilot test is necessary to identify potential problems of the where n is the number of criteria j under an objective i.
instrument and provide a preview of how difficult the questions are to
complete. Stage five: Web strategy consistency analysis.
Step 1: Gap value analysis for each criterion. For purposes of
improvement, managers should give attention to criteria with low
average scores. However, in considering scores only as the basis for
improvement, managers may be misled into allo-cating more resources
to low-scoring criteria. To generate a more applicable assessment result,
Stage three: Execution of Web site evaluation. a gap (Gij) is calculated by
Step 1: Selecting a panel of experts as evaluators. As mentioned
earlier, this paper aims to evaluate a Web site from the internal
perspective of an organization to ensure that strategy is consistent with
G ¼ AS W
ij ij ij (5)
Web site presence. Law et al. (2010) also suggested that consumers do where i is an objective and j is its related criteria.
not have sufficient insights into how Web site performance is accurately The gap of each criterion is then compared with a threshold, which is
measured. For these reasons, the expert-based evaluation decided by the manager, to determine whether such criterion is inconsistent
with strategy. When the absolute value of
 W.-C. Chiou et al. / Tourism Management 32 (2011) 1463e1473 1467

Fig. 2. Criteria performance matrix chart.

the gap is greater than the threshold, the criterion is recognized as Fig. 3. Hierarchical evaluation structure and criteria weights of L site.
inconsistent with strategy. However, a gap threshold is a subjective
value and is adjustable based on the available resources and main are added to compute the average dimensional weight (AWd)
concerns of a company.
and dimensional score (ASd) using the following equations:
Step 2: Constructing a criteria performance matrix chart. A n
matrix chart is used to provide managers with criteria perfor- P
j ¼ 1 Wdj
mance information in graphical form. The matrix is also used in AWd ¼ n (6)
the priority ranking of improvement plans for strategy-incon- n
P
sistent criteria. The criteria performance matrix (Fig. 2) origi- j ¼ 1 ASdj
nated from the service quality performance matrix (Hung, ASd ¼ n (7)
Huang, & Chen, 2003). A criterion presence score is plotted
against the x-axis and the corresponding criterion importance where d is a 4PsC dimension (d ¼ 1e5), j is a criterion number,
(weight) is plotted against the y-axis. The scale of both axes is and n is the number of criteria under the 4PsC dimension.
between 0 and 1. The matrix chart has nine cells when the axes
are equally divided into three sections with four scales (i.e., 0, 1/
3, 2/3, 1). Two off-diagonal lines, which are subjectively
Goal Objectives Criteria (weight)
H
adjustable according to the organization’s available resources, 1. Product
1. Product variety (0.91)
H, T
fi Variety 2. Customized offerings (0.64) H
are added as a con dence interval to make the objective zone (T) 3. Valuable bundle or product suggestion (0.77)

more reasonable. The manager must determine a proper

F, T
confidence level before interpreting the results. The objective 1. Quick response to customer (0.91)
2. Interactive communications (0.64) F, T
zone (T) is defined as the area between two off-diagonal lines. A 2. Quality 3. Product quality (0.91) F
F
criterion located in the objective zone is consistent with strategy Service 4. Delivery product as promised (0.91)
F, T
under a level of confidence and should therefore be maintained. 5. Customer service support (0.91)
F, T
A criterion located in the improve zone (T1)dsuggesting that 1. Product search or assortment (0.91)
H
the corresponding criterion presence score is far behind its 2. Hierarchical product category (0.77)
importance weightdis considered inconsistent with strategy Customer 3. Ease of online transaction (0.77) H, T
satisfaction 3. Convenient 4. User-friendly interface (0.77) H, T
and should therefore be improved. To some extent, a criterion Shopping 5. Ease of navigation (0.77) H, T
F, T
located in the reduce zone (T2) is deemed consistent with 6. Order status inquiry and tracking (0.77)
F, T
strategy. However, managers should consider reducing the 7. Easy to cancel or modify order (0.77)
F, T
resource allocation for this criterion because its presence score 1. Reliable and innovative system (0.77)
far exceeds the importance weight (i.e., the criterion is over- 4. Innovative 2. Data retrieve mechanism (0.77) F, T
Technology 3. Online assistance and help (0.64) F, T
performing). Criteria located at the top-left corner or the
F
bottom-right corner (the darkest area) need improvement for 1. Promotion campaign (0.91)
better scores and reduction in resource allocation. 5. Best Deals 2. Advertising and banner (0.64) F
3. Competitive price (0.91) F

Step 3: Analyzing 4PsC dimensions and transactional process

F
performance. The selected criteria are classified into the 4PsC search finding provided; H hyperlink provided; T technology supported
dimensions following the criteria pool classification in Table 1.
Each dimensional criteria weight (Wdj) and average score (ASdj) Fig. 4. Hierarchical evaluation structure and criteria weight of Z site.
1468 W.-C. Chiou et al. / Tourism Management 32 (2011) 1463e1473

A radar chart is then constructed to identify the strengths and weaknesses n

P
W
of the 4PsC dimensions. By comparing the weights and scores of each j¼1
tdj
$

dimension, we can identify the worst dimension, which is very important, but AWtd ¼ n (8)
with poor presence. To analyze further the cause of the poor presence of the
dimension, a drill-down analysis can help managers discover the related criteria n AS
P j¼1 tdj
$
with a relatively low score. Web customers leave a Web site without AStd ¼ n (9)
completing a transaction for many reasons. The analysis of each transactional
phase can help businesses investigate which weak phase results in the loss of where t is the transactional phase (t ¼ 1e3), d is a 4PsC dimension (d ¼
Web customers. To examine criteria presence in each transactional phase, the 1e5),
criteria (j) are first clas-sified into one of three transactional phases (t) based on j is the criterion number (j ¼ 1wn), n is the total criterion number under the
discussions with the manager and the definition of each phase. Subsequently, 4PsC dimension in each phase, Wtdj is the weight of criterion j under a
the criteria are further grouped into the 4PsC dimensions (d) following the dimension d in phase t, and AStdj is the average score of criterion j under a
criteria pool classification in Table 1. In this way, the average 4PsC dimensional dimension d in phase t.
weight (AWtd) and average score (AStd ) in each phase can be calculated
following Eqs. (8) and (9), respectively.
4. Strategic evaluation of two Taiwan online travel sites

To demonstrate how the proposed evaluation framework can be

implemented, we selected two leading travel companies in Taiwan

Fig. 5. Hyperlink to payment method Web page of L site.

 W.-C. Chiou et al. / Tourism Management 32 (2011) 1463e1473 1469

as study cases. L site has been in the traditional travel business for over thirty Table 3
years and has been providing e-services to-consumers since 2000. The Weights, weighted scores, and gaps of Z site.

company is both a travel agent and a supplier of travel products, such as flight W G
Objectives (O i) Criteria (C ij) ij AS ij ij NWij WSij OWSi
tickets, accommodations, and group tour packages, to other agencies. The 1. Product Variety 1. Product variety 0.91 0.70 0.21 0.39 0.27 0.68
company’s Web site is designed for the implementation of business-to-business 2. Customized offerings 0.64 0.64 0.00 0.28 0.18
(B2B) and business-to-consumer (B2C) Web strategies. 3. Valuable bundle 0.77 0.70 0.07 0.33 0.23
or suggestion

Another selected company is Z site. It has been in the travel business for 2. Quality Service 1. Quick response 0.91 0.66 0.25 0.21 0.14 0.67
nine years and was originally an online travel store. The company generally to customer
2. Interactive 0.64 0.70 0.06 0.16 0.11
targets young clienteles and families who intend to travel without tour guides.
communications
According to a survey by InsightXplorer Limited(2006), Z site is the most well 3. Product quality 0.91 0.64 0.27 0.21 0.14
known and frequently visited travel site in Taiwan. It also had the highest 4. Delivery product 0.91 0.64 0.27 0.21 0.14
revenue and growth rate in Taiwan’s online travel sector in 2006. These two as promised
companies have different business strategies for target customers and business 5. Customer 0.91 0.64 0.27 0.21 0.14
service support
operations; therefore, the evaluation results should provide good implications
and comparisons. 3. Convenient 1. Product search 0.91 0.72 0.19 0.16 0.12 0.69
Shopping or assortment
2. Hierarchical 0.77 0.71 0.06 0.14 0.10
product category
4.1. Stage one: identification of web site strategy and criteria 3. Ease of online 0.77 0.70 0.08 0.14 0.10
transaction
4. User-friendly 0.77 0.70 0.08 0.14 0.10
After conducting an in-depth interview with L site’s vice-general manager interface
on October 14, 2008, the goal of L site was defined as “one-stop shop with 5. Ease of navigation 0.77 0.63 0.14 0.14 0.09
quality service”. To achieve this goal, six objectives are set: superior product 6. Order status 0.77 0.68 0.09 0.14 0.09
inquiry and tracking
line, convenient shopping, content rich-ness, interactivity (Web 2.0), customer
7. Easy to cancel 0.77 0.66 0.11 0.14 0.09
service, and trust. A hier-archical evaluation structure was constructed (Fig. 3) or modify order
and sent back to the manager for confirmation. Fig. 4 shows the relationship
4. Innovative 1. Reliable and 0.77 0.70 0.07 0.35 0.25 0.69
between the goal, objectives, and criteria of L site. While the Technology innovative system
2. Data retrieve 0.77 0.68 0.09 0.35 0.24
mechanism
Table 2 3. Online assistance 0.64 0.68 0.04 0.30 0.20
Weights, weighted scores, and gaps of L site. and help
W AS G NW WS
Objectives (O i) Related Criteria (C ij) ij ij ij ij ij OWSi 5. Best Deals 1. Promotion campaign 0.91 0.72 0.19 0.37 0.27 0.70
1. Superior 1. Product variety 0.91 0.75 0.16 0.27 0.20 0.66 2. Advertising 0.64 0.70 0.07 0.26 0.18
product line 2. Promotion campaign 0.77 0.67 0.11 0.23 0.15 and banner
3. Product quality 0.77 0.65 0.13 0.23 0.15 3. Competitive price 0.91 0.68 0.23 0.37 0.25
4. Customized offerings 0.91 0.60 0.31 0.27 0.16
2. Convenient 1. Product search 0.91 0.74 0.17 0.33 0.25 0.71
Shopping or assortment hierarchical evaluation structure was being confirmed, the vice-general
2. Convenient 0.91 0.68 0.23 0.33 0.23 manager of L site was asked to assess the importance of each criterion using
payment methods linguistic terms; the assessments were then transformed into crisp numbers. For
3. Ease of online 0.91 0.70 0.21 0.33 0.23
transaction instance, C11 (product variety)
3. Content 1. Product details 0.77 0.66 0.11 0.33 0.22 0.68
richness 2. Comprehensive 0.77 0.72 0.05 0.33 0.24
content coverage

3. Advertising and banner 0.77 0.66 0.11 0.33 0.22

4. Interactivity 1. Interactive 0.64 0.70 0.06 0.29 0.20 0.68
communications
2. Online assistance 0.77 0.64 0.13 0.35 0.23
and help

3. Member community 0.77 0.72 0.05 0.35 0.25

5. Customer 1. Quick response 0.91 0.70 0.21 0.26 0.18 0.70
service to customer
2. Order status 0.91 0.71 0.19 0.26 0.19
inquiry and tracking
3. Easy to cancel 0.77 0.68 0.09 0.22 0.15
or modify order
4. Customer 0.91 0.70 0.21 0.26 0.18
service support

6. Trust 1. Reliable and 0.77 0.52 0.26 0.19 0.10 0.66

innovative system
2. Reputation 0.77 0.68 0.09 0.19 0.13
and credibility
3. Delivery product 0.77 0.63 0.14 0.19 0.12
as promised
4. Security protection 0.91 0.74 0.17 0.22 0.16
5. Purchasing guarantee 0.91 0.69 0.22 0.22 0.15
Fig. 6. Criteria performance matrix of L site.
1470 W.-C. Chiou et al. / Tourism Management 32 (2011) 1463e1473

Table 5
4PsC dimensional average weights and scores of Z site.

Dimensions (d) Related Criteria (Cij) AWd ASd

C ,C ,C ,C ,C ,C ,C ,C ,C
1. Place 31 32 33 34 35 37 41 42 43 0.77 0.68
C ,C
2. Product 11 23 0.91 0.67
C
3. Price 53 0.91 0.68
C ,C
4. Promotion 51 52 0.77 0.71
C ,C ,C ,C ,C ,C ,C
5. Customer Relationship 12 13 21 22 24 25 36 0.79 0.67

industry, whereas the other four are owners and directors of local travel
agencies. All the evaluators are well experienced in online travel transactions.
Before evaluating each questionnaire, the evaluators identified themselves and
read the goals and objectives of the two Web sites. While evaluating the
questionnaires, evalu-ators could click either the hyperlink or the finding button
at the end of each questionnaire, leading them to relevant Web pages or
findings. Subsequently, evaluators used linguistic terms to express their
agreement or disagreement with questionnaire statements.

4.4. Stage four: criteria weights and score calculation

Fig. 7. Criteria performance matrix of Z site. We take the C11 (product variety) criterion of L site in Table 2 as a
calculation example. The criterion weight (W11) assigned by the manager is
was assessed as “very important,” and the crisp number 0.91 was then assigned
0.91. The normalized weight (NW11 ¼ 0.27) was derived using Eq. (1); we
to W11 as suggested by Chen and Hwang (1992). Another interview with Z divided the criterion weight (0.91) by the total weight (3.36) of the four criteria
site’s marketing and IS manager was conducted on December 6, 2008. under O1 (superior product line). The average score 0.75 (AS11) of C11 was
Accordingly, the goal of Z site is “customer satisfaction”. The site’s hierarchical
calculated using Eq. (2). Using Eq. (3), we obtained the weighted score (WS11
evaluation structure is shown in Fig. 4. The criteria weights (Wij) for L and Z
¼ 0.20). The objective weighted score was then derived (OWS1 ¼ 0.66) using
sites are listed at the end of each criterion (Figs. 3 and 4, respectively). Eq.
The two sites substantially differ in several objectives. L site does not (4). The details of the criteria weights and scores for L and Z Web sites are
consider providing a “competitive price” to customers as a strategic criterion listed in Tables 2 and 3, respectively.
mainly because the company sells products not only to customers but to other
travel agents as well. Hence, they need to reserve some margin for agents. In
4.5. Stage five: Web strategy consistency analysis
contrast, Z site is a pure online retailing shop, and one of its objectives is to
provide the “best deals” to customers. Hence, C53 (competitive price) is The gaps (Gij) listed in Tables 2 and 3 were calculated using Eq.
included in the criteria list. (5). Results showed that most of the criteria of the two Web sites have negative
values. In theory, a negative gap shows that the criterion importance weight
exceeds the corresponding presence score; hence, the criterion is deemed
4.2. Stage two: Web-based evaluation instrument development inconsistent with strategy. To determine the gap thresholds, Tables 2 and 3
were reviewed by the managers of L and Z sites, and the values were set as 0.23
Identified conceptual criteria for the two Web sites were inter-preted in a and 0.25, respectively. When a criterion’s absolute gap is greater than the
way that is relevant to them and to the tourism industry as a whole. threshold, the criterion is considered inconsistent with strategy.
Consequently, 22 questions were developed for L site, whereas 21 questions
were developed for Z site. An evaluation-supporting tool was designed and
provided at the end of each questionnaire as either a hyperlink or a finding. For
instance, the question “Does the Web site provide a convenient payment
method?” for L site has a hyperlink to the Web page of the payment method.
The captured screen of L site’s Web page is shown in Fig. 5.

4.3. Stage three: execution of Web site evaluation

A panel of seven experts evaluated the criteria presence ratings. Three of

the evaluators are e-business scholars in the tourism

Table 4
4PsC dimensional average weights and scores of L site.

Dimensions (d) Related Criteria (Cij) AWd ASd

C ,C ,C ,C ,C ,C ,C ,C
1. Place 21 22 23 32 42 61 62 64 0.84 0.68
C ,C ,C
2. Product 11 13 31 0.82 0.69
3. Price N.A. N.A. N.A.
C12, C33, C53, C65
4. Promotion 0.67 0.81
C ,C ,C ,C ,C ,C ,C
5. Customer Relationship 14 41 43 51 52 54 63 0.83 0.68
Fig. 8. 4PsC dimensions performance of L site.
 W.-C. Chiou et al. / Tourism Management 32 (2011) 1463e1473 1471

Although C61 (reliable and innovative system) is still inside the objective zone,
it is nearly located in the improve zone and should be closely watched. The
same implication applies to Z site (Fig. 7). There are only three criteria weights
that appear in both matrices because the weights were assigned by only one
manager for each Web site. Both managers considered the selected criteria as
“partly important”, “important”, and “very important”. Thus, only three crisp
values were transformed accordingly. Broadly speaking, most criteria of the
two Web sites are located in the objective zone, which means that the sites are
generally consistent with their strategies. Z site seems to have a better
performance than L site because its criteria scores are located near the center
of the objective zone.

To identify the weaknesses and strengths of each 4PsC dimen-sion by radar

chart, the average dimensional weights (AWd) and average dimensional scores
(ASd) were calculated using Eqs. (6) and
(7), respectively. The results are summarized in Tables 4 and 5. As shown in
Fig. 9. 4PsC dimensions performance of Z site. Fig. 8, L site does not have all 4PsC dimensions; the “price” dimension is left
out. This is because the site does not emphasize a pricing strategy and no
pricing criterion is selected. The figure shows that the presence scores of the
Based on the gap threshold, the three worse criteria of L site are C14
four dimensions are generally less than the weights by about 0.1 point. From a
(customized offerings; 0.31), C61 (reliable and innovative system; 0.26), and “product” stand-point, for example, the company may need to make additional
C22 (convenient payment methods; 0.23). These criteria should be prioritized efforts on low-scoring criteria, such as C13 (product quality) and C31 (product
in action plans for improve-ment. For Z site, four criteria reach the gap detail).
threshold: C23 (product quality; 0.27), C24 (deliver product as promised; 0.27),
C25 (customer service support; 0.27), and C21 (quick response to customers; On the other hand, the evaluation of Z site entails a five-dimension analysis
0.25). Although the two Web sites have a few criteria with positive gap values, (Fig. 9). The manager regards “product” and “price” as very important
no reduction in resource allocations should be taken because, as indicated by dimensions. Both have the same weight of 0.91, but the scores of these two
the managers, these gaps are reasonable. dimensions are moderate (0.67 and 0.68, respectively). When compared, the
gaps between the scores and weights of these two dimensions are 0.23 and 0.24,
To provide managers with a graphical analysis, a criteria performance respectively, which are much higher than the gaps of the other three dimensions
matrix was introduced to identify criteria located outside the objective zone. ( 0.09, 0.12, and 0.06). Therefore, the manager should exert extra efforts in
Compared with the gap value method, the criteria performance matrix provides improving low-scoring criteria under the “product” and “price” dimensions,
a better overview of criteria performance and can easily identify critical criteria such as C23 (product quality) and C53 (competitive price).
that must be carefully watched. In Fig. 6, C14 (customized offerings) is located
outside the objective zone and must be improved. The graphical analysis is
consistent with the previous gap value analysis showing that the criterion has To analyze the presence scores of 4PsC dimensions in three different
the largest negative gap value ( 0.31) and is considered as the first priority for transactional phases, we classified the dimensional criteria into these phases.
improvement. The average 4PsC dimensional weights (AWtd) and scores (AStd) in each
phase were calculated using Eqs. (8)

Goal: One-stop shop with quality service

Product Promotion Product Promotion Product Promotion

C.R. C.R. C.R.

Place Price Place Price Place Price

Information Phase Agreement Phase Settlement Phase

1. Information 2. Agreement 3. Settlement

Criteria AWtd AStd Criteria AWtd AStd Criteria AWtd AStd
C ,C C ,C ,C C ,C ,C
1.Place 21 32 0.84 0.73 23 42 61 0.82 0.62 22 62 64 0.86 0.70
C ,C ,C
2.Product 11 13 31 0.82 0.69 N/A N/A
C ,C ,C ,C
3.Promotion 12 33 53 65 0.81 0.67 N/A N/A
C ,C C C ,C ,C ,C
4.C.R. 14 51 0.91 0.65 41 0.64 0.70 43 52 54 63 0.84 0.69

Fig. 10. L site’s 4PsC dimensional average weights and scores in three phases.
1472 W.-C. Chiou et al. / Tourism Management 32 (2011) 1463e1473

Goal: Customer satisfaction

Product Promotion Product Promotion Product Promotion

C.R. C.R. C.R.

Place Price Place Price Place Price

Information Phase Agreement Phase Settlement Phase

1. Information 2. Agreement 3. Settlement

AW AS AW AS AW AS
Criteria td td Criteria td td Criteria td td
C ,C ,C ,C ,C C ,C ,C C
1.Place 31 32 34 35 42 0.80 0.69 33 41 43 0.73 0.69 37 0.77 0.66
C ,C
2.Product 11 26 0.91 0.67 N/A N/A
C
3.Price 53 0.91 0.68 N/A N/A
C ,C
4.Promotion 51 52 0.78 0.71 N/A N/A
C ,C ,C C C ,C ,C
5.C.R. 12 13 21 0.77 0.67 22 0.64 0.70 24 25 36 0.86 0.65

Fig. 11. Z site’s 4PsC dimensional average weights and scores in three phases.

and (9), respectively. As shown in Fig. 10, “place” is the best-per-forming presence. Lastly, “C. R.” is the worst dimension, with a low score of 0.65 in the
dimension of L site with the highest average score of 0.73 during the settlement phasedbecause the site performs poorly in delivering products as
information phase, whereas “C. R.” is the worst-per-forming dimension with a promised and in providing good customer service support.
low average score of 0.65. Criteria under the “place” dimension mostly relate
to Web site design and infor-mation technology. A user-friendly Web site can
help customers find target information easily and decide quickly. In exploring
the reason for the low “C. R.” score, we found that C14 (customized offerings) 5. Conclusions and suggestions
was rated with an average score of 0.6 (Table 2, AS14) by the seven evaluators.
This criterion can be improved by offering customers more differentiated travel Very limited research explores the issue of Web strategy in Web site
products and flexibility, enabling them to customize their own tour plans. Note evaluation, and none includes Web strategy in their evaluation frameworks. In
that pricing is not considered by L site; hence, “price” is not presented in the response to this, a strategic framework was adopted to ensure consistency
information phase. In the information phase, the manager considers “C. R.” as between Web strategy and actual Web site presence. To select relevant criteria
the most important dimension (the darkest shade) with a weight of 0.91, but the from the proposed criteria pool, a strategic framework was developed based on
score of this dimension is only 0.65. In this phase, this dimension is regarded
the goals and objectives of Web sites. To implement this framework, we intro-
as inconsistent with strategy. From the viewpoint of the agreement phase, the
“place” dimension has the lowest average score (0.62). To address further this duced a five-stage evaluation process, serving as a systematic approach to
strategic Web site evaluation.
issue, two problem criteria were identified: C42 (online assis-tance and help)
and C61 (reliable and innovative system). These criteria have low scores of To illustrate how a Web site evaluation is conducted using a strategic
0.64 (Table 2, AS42) and 0.52 (Table 2, AS61), respectively. framework, this study selected two leading travel Web sites in Taiwan as
demonstration cases. Two most telling findings emerge from the study. First, L
and Z sites are apparently different in their Web site strategies; hence, criterion
selection not only varies in the number of criteria but also in criterion attributes.
Fig. 11 shows the dimensional scores of Z site in the three transactional Figs. 3 and 4 clearly demonstrate differences in the evaluation frameworks of
phases. Compared with L site, Z site includes “price” as one of the five- these sites. Second, our Web strategy consistency analysis assists managers in
identifying inconsistent areas and in improving such areas. For instance, L site
dimensions in the information phase; C53 (competitive price) is considered a
has the goal “one-stop shop with quality service,” and the objective “convenient
very important criterion in the site’s pricing strategy. “Promotion” is the best- shop-ping” is regarded as one of its relevant objectives. We examined whether
performing dimen-sion with a high score of 0.71, which is different from L
this objective is consistent with the goal. Table 2 indi-cates that this objective
site’s best-performing dimension, that is, “place,” in the information phase. This
is in line with the goal because it obtained the highest score (0.71).
can be explained by the fact that Z site is a pure online travel company and
Nevertheless, the manager can still introduce improvements to this objective
competitive pricing is a key factor in attracting online customers. Periodically
by providing more convenient payment methods. Z site, on the other hand,
introducing attractive promotion campaigns is important for Z site to create
argues that “quality service” is a necessary objective in attaining its goal, which
more sales because the site does not sell products to other travel agencies,
is “customer satisfaction.” Unfortunately, this objective obtained the lowest
unlike L site. Indeed, the manager considers “product” and “price” as very
score (0.67). To remedy this inconsistency issue, the manager can introduce
important dimensions (the darkest area) in this phase, but the scores of these
two dimensions are low. It appears that in these two dimensions, an improvements to relevant low-scoring criteria in Table 3, such as C21 (quick
inconsistency exists between Web site strategy and response to
 W.-C. Chiou et al. / Tourism Management 32 (2011) 1463e1473 1473

customer), C23 (product quality), C24 (deliver product as prom-ised), and C25 References
(customer service support).
We introduced an evaluation-supporting tool to provide a context reference Chen, S. J., & Hwang, C. L. (1992). Fuzzy multiple attribute decision making-method and
for evaluators through a hyperlink of relevant Web page or supporting materials application. New York: Springer-Verlag.
Chiou, W. C., Lin, C. C., & Perng, C. (2010). A strategic framework for website eval-uation based
with respect to specific ques-tions. By doing so, evaluators can easily and on a review of the literature from 1995e2006. Information & Management, 47(5/6), 282e290.
quickly focus on target Web pages or refer to facts. Nevertheless, evaluators Chiou, W. C., Perng, C., Tsai, J. T., & Lin, C. C. (2008). The review of the website evaluation
must bear in mind that this supporting tool acts as a decision-making reference. framework in IS and marketing journals. International Journal of Information Systems &
Change Management, 3(1), 81e104.
To make objective evaluations, evaluators are free to explore more Web pages
Choi, S., Lehto, X. Y., & Oleary, J. T. (2007). What does the consumer want from a DMO
or online transaction processes when necessary. To improve the quality of the Website? A study of US and Canadian tourists’ perspectives. Interna-tional Journal of
supporting tool, researchers or practi-tioners may collect suggestions from Tourism Research, 9, 59e72.
Clyde, L. A. (2000). A strategic planning approach to Web site management. The Electronic
evaluators to redesign the hyperlink or finding on relevant questions.
Library, 18(2), 97e108.
Evans, J. R., & King, V. E. (1999). Business-to-business marketing and the World Wide Web:
This research has a twofold contribution. First, the strategic framework planning, managing, and assessing web sites. Industrial Marketing Management, 28(4),
helps managers easily develop individual Web site evaluation criteria based on 343e358.
Horng, J. S., & Tsai, C. T. (2010). Government websites for promoting East Asian culinary
their strategies. The strategic frame-work was designed to focus on how a tourism: a cross-national analysis. Tourism Management, 31(1), 74e85.
specific Web site applies its goals and objectives, not for an industry or the Hung, Y. H., Huang, M. L., & Chen, K. S. (2003). Service quality evaluation by service quality
Internet in general, as designed in previous studies. Second, the five-stage performance matrix. Total Quality Management, 14(1), 79e89.
InsightXplorer Limited. (2006). Over 90% of internet users love traveling and prefer to plan their
evaluation process assists manager in effectively discovering strategy- own trip. available http://www.insightxplorer.com/news/news_ 11_22_05.html.
inconsistent criteria through a combination of qualitative and quantitative
methods. Law, R., & Leung, R. (2000). A study of airlines’ online reservation services on the Internet.
Journal of Travel Research, 39, 209e211.
Law, R., Qi, S., & Buhalis, D. (2010). Progress in tourism management: a review of website
There were two limitations of this study. First, the 4PsC dimen-sions apply evaluation in tourism research. Tourism Management, 31(3), 297e313.
to commercial Web sites but not to non-commercial Web sites. The “price” Morrison, A. M., Taylor, J. S., & Douglas, A. (2004). Website evaluation in tourism and
hospitality: the art is not yet stated. Journal of Travel & Tourism Marketing, 17(2/3),
dimension may not be relevant to non-commercial Web sites; it can therefore
233e251.
be removed accordingly. Second, the evaluation criteria pool may need to be Park, Y. A., & Gretzel, U. (2007). Success factors for destination marketing web sites:
revised by col-lecting relevant criteria with respect to non-commercial Web a qualitative meta-analysis. Journal of Travel Research, 46, 46e63. Robbins, S.
P. (1994). Management (4th ed.). New York: Prentice-Hall Inc.
sites. Furthermore, rapid developments in information technology have
Roney, S. A., & Özturan, M. (2006). A content analysis of the web sites of Turkish travel
introduced a plethora of new terminologies. The criteria listed in Table 1 are agencies. An International Journal of Tourism and Hospitality Research, 17(1), 43e54.
conceptual and can be further interpreted to fit specific industries and individual Saaty, T. L. (1980). The analytic hierarchy process. McGraw Hill Publications. Standing, C., &
contexts. Vasudavan, T. (2000). The marketing of regional tourism via the
internet: lessons from Australian and South. Marketing Intelligence & Planning, 18(1),
45e48.
As suggested by Law et al. (2010), seeking the views of industrial Stepchenkova, S., Tang, L., Jang, S. C., Kirilenko, A. P., & Morrison, A. M. (2010).
practitioners and consumers remains important because these groups are the Benchmarking CVB website performance: spatial and structural patterns. Tourism
Management, 31(5), 611e620.
ultimate suppliers and users of tourism Web sites. We agree that an external Thorn, K., & Chen, H. C. (2005). E-business in the New Zealand tourism industry: an
evaluation is still necessary in assessing the effectiveness of Web sites from the examination of implementation and usage. Current Issues in Tourism, 8(1), 39e61.
consumer viewpoint. This should be conducted after the strategy consistency Tsai, W. H., Chou, W. C., & Lai, C. W. (2010). An effective evaluation model and
improvement analysis for national parks websites: a case study of Taiwan. Tourism
of Web sites is confirmed through our proposed internal evaluation process. If Management, 31(6), 936e952.
the two evaluation results show significant differences in particular criteria, Zadeh, L. A. (1995). Probability theory and fuzzy logic are complementary rather than
managers may need to review their strategies and clarify the causes of such competitive. Technometrics, 37(3), 271e276.
Zhou, Q., & DeSantis, R. (2005). Usability issues in city tourism website design: a content
discrepancies.
analysis. In Proceedings of 2005 IEEE International Professional Communication
Conference. Available http://ieeexplore.ieee.org/xpl/freeabs_ all.jsp?arnumber¼1494253.