Interdisciplinary Journal of E-Learning and Learning Objects Volume 6, 2010

Developing Web-Based Learning Resources in

School Education: A User-Centered Approach
Said Hadjerrouit
University of Agder, Kristiansand, Norway
Said.Hadjerrouit@uia.no

Abstract
Web-based learning resources (WBLRs) are potentially powerful tools for enhancing teaching
and learning processes in school education. They can provide teachers and learners with a wide
range of new and exciting experiences that are not possible in a traditional classroom. However,
WBLRs are still the domain of technical and software experts rather than teachers and learners.
As a result, much of the development of WBLRs is carried out without a true understanding of
issues pertinent to learning and pedagogy. Also lacking is user involvement in the development
process of WBLRs. The aim of this work is to propose a user-centered approach to the develop-
ment of WBLRs to translate pedagogical issues into a software tool that supports effective learn-
ing. The article also reports on the application of the approach in school education.
Keywords: Pedagogical usability, user-centered development approach, Web-based learning
resources, WBLRs

Introduction
The potential added value of Web-based learning (or similar designations, such as “virtual learn-
ing”, “technology-based learning”, or “online learning”) compared to teacher- and textbook-based
instruction lies in helping learners to acquire the right knowledge and skills in order to function as
active, self-reflected, and collaborative learners (Govindasamy, 2002; Hamid, 2002). However,
this cannot be realized without a change from learning environments in which the teacher and the
textbook structure the learning process, towards learning environments in which the students
themselves control, under the guidance of the teacher, the order in which they learn and perform
activities based on their needs (Erstad, 2006; Wilson, 1998). Web-based learning resources
(WBLRs) have the potential to support a learning environment in which students explore knowl-
edge and enhance their learning (Combes & Valli, 2007). To realize this, the development of
WBLRs needs to be user-centered. User-centered design is an approach that puts the intended
users of WBLRs at the centre of its design and development (Winograd, 1996). In school educa-
tion, students are considered as the most important users of WBLRs, in addition to teachers as
guides and facilitators of learning.
Material published as part of this publication, either on-line or The remainder of this article is struc-
in print, is copyrighted by the Informing Science Institute.
Permission to make digital or paper copy of part or all of these
tured as follows. First, the paper under-
works for personal or classroom use is granted without fee takes a literature review to provide an
provided that the copies are not made or distributed for profit understanding of the concept of WBLR,
or commercial advantage AND that copies 1) bear this notice on the one hand, and development issues
in full and 2) give the full citation on the first page. It is per- of WBLRs, on the other hand. Second,
missible to abstract these works so long as credit is given. To
copy in all other cases or to republish or to post on a server or concepts of usability are analyzed, and a
to redistribute to lists requires specific permission and payment definition of pedagogical usability is
of a fee. Contact 0HPublisher@InformingScience.org to re- proposed. Third, the paper outlines a
quest redistribution permission. user-centered development approach to

Editor: Thomas Connolly

Developing Web-Based Learning Resources

WBLRs. An application example of the approach is then presented. This is followed by the
evaluation of the approach. Finally, some remarks on further work conclude the article.

Research Goal
This work is the second major part of a research project that aims to develop and evaluate
WBLRs in school education. The first part of the work aims to review the literature to develop a
framework of critical elements in WBLR evaluation (Hadjerrouit, 2010). A brief summary of the
evaluation results are described in the section “Users’ Perceptions of WBLRs.”
The aim of this paper is to propose a user-centered approach to the development of WBLRs,
along with the study of three WBLRs to report on critical elements in the development of
WBLRs. Traditionally, user-centered design is defined as an approach to software design that
grounds the process in information about the users of the software product. User-centered design
processes focus on users through the analysis, design, implementation, and evaluation of the
product (Winograd, 1996). Basically, user-centered design focuses on technical usability as de-
fined by Nielsen (1993, 2000). Technical usability is a self-evident requirement, but it is limited
when it comes to the development of WBLRs because it does not emphasize pedagogical usabil-
ity, which is of crucial importance in school education. Hence, developers of WBLRs need to
incorporate pedagogical considerations in the development process. It is also important to identify
the users of WBLRs and the conditions under which they use them. The major users of WBLRs
are school students, who use the resources to achieve learning goals under the guidance of the
teachers. They use them under classroom conditions and within the school context.

Literature Review
The Concept of WBLR
A closer look at the research literature shows that the concept of WBLR is similar to the term
“Web-based learning tools”, also referred to as “learning objects”, found in Kay and Knaak,
(2005, 2008) and Kay, Knaak, and Petrarca (2009). The term is defined as “interactive Web-
based tools that support learning by enhancing, amplifying, and guiding the cognitive processes
of learners.” Moreover, WBLRs include the main features of the term “Web-based learning ap-
plication” that is defined by Liu & LaMont Johnson (2005) as instructional content or activity
delivered through the Web that teaches a focused concept, meets specific learning objectives,
provides a learner-centered context, and is an individual and reusable piece. Accordingly, the
concept of WBLR can be defined as a learning object or Web-based learning tool with four major
features:
a) It uses Web technologies and is delivered through the Web
b) It teaches content that meets specific learning objectives aligned with the curriculum
c) It is designed on the basis of a learning strategy and pedagogical procedure
d) It contains reusable elements
From a technological point of view, WBLRs use Web technologies and Internet services as the
delivery mode, that is to say HTML, URL, browsers, e-mail, file transfer facilities, etc. In addi-
tion to scripting languages, such as PHP and JavaScript, WBLRs incorporate multimedia ele-
ments, such as animations, video and audio clips, images, graphics, and those developed with
multimedia authoring software, such as Authorware, Micromedia Flash, and Hot Potatoes. From a
pedagogical point of view, WBLRs are embedded within a learning strategy linked to the cogni-
tivist, constructivist, or collaborative learning paradigm or a combination of them (Martinidale,
Cates, & Qian, 2005). Hence, WBLRs are associated with pedagogical values that potentially af-
fect teaching and learning processes in school education. From the content point of view, WBLRs

116
 Hadjerrouit

are computer-based implementations of a specific subject that is normally aligned with a given
curriculum in school education. WBLRs can be created to support different topics of a given sub-
ject, as well as learning material in a number of subject areas at all levels in school education.
Summarizing, the core of WBLRs is the integration of content, technology, and pedagogy into a
system that supports learning. With other words, WBLRs exist at the intersection of content, pe-
dagogy, and technology (Figure 1).

Figure 1: Web-based learning resources: Main characteristics

Finally, WBLRs need to be reusable in order to satisfy the user’ needs (Johnson & Hall, 2007).
Reusability is useful for learning school subjects in different educational settings. It assumes that
elements of WBLRs can be found to fit into another or new lesson (Strijker & Collis, 2007). Re-
usability also assumes that a given lesson or course will find WBLRs or elements of them from
many online resources or throughout a database repository.

Development Issues
Due to the increased focus on digital literacy, WBLRs as pedagogical tools are becoming more
and more important in school education (ITU Monitor, 2009). However, the research literature is
just beginning to report on the development of WBLRs in educational settings (Combes & Valli,
2007), and there are several issues that need to be addressed. First, WBLRs are still the domain of
technical and software experts rather than teachers and learners (Nam & Smith-Jackson, 2007).
As a result, most WBLRs basically emphasize technical usability as defined by Brinck, Gergle,
and Wood (2002) and Nielsen (2000). Technical usability is important to minimize the cognitive
load, and helps to free more resources for the learning process itself. It enables learners to easily
focus on learning materials without having to make an effort to figure how to access them. How-
ever, although technical usability is a self-evident requirement for WBLRs, it is not necessarily
conductive for deep learning. For instance, according to Ingram (2003), the degree and rapidity of
navigation, which is a technical usability criterion, does not necessarily measure the quality of the
WBLR in terms of learning. Clearly, navigation is supposed to measure task performance, not
learning, since the aim is to determine whether learners can effectively navigate through the
WBLR, for example the number of links learners had to follow to find specific information.
Second, to support student learning, a number of researchers (Kukulska-Hulme & Shield, 2004;
Laurillard, 2002; Leacock & Nesbit, 2007; Nokelainen, 2006) suggest that developers need to
design WBLRs with suitable usability, and then tailor it more closely to meet the learners’ needs,
thereby implying that there is a further dimension to consider when designing WBLRs. This di-

117
Developing Web-Based Learning Resources

mension is often called “pedagogical usability” as it is associated with aspects that are fundamen-
tal to learning. Hence, to deal with pedagogical issues, technical usability must be extended to
capture elements that are pertinent to learning. However, little attention has been paid to peda-
gogical usability of WBLRs, which is a critical success factor for the use of this technology in the
classroom. From the viewpoint of pedagogical usability, current WBLRs lack a number of fea-
tures that would make them more interactive, motivating, and collaborative.
Third, the research literature reveals that WBLRs with advanced features are difficult to design,
and therefore current systems are still limited in their pedagogical usability. Martinidale, Cates,
and Qian (2005) stated that it is substantially more difficult to create WBLRs that accommodate
the demands of constructivist learning. Likewise, Liu and LaMont Johnson (2005) found a lack of
fit between existing WBLRs and what teachers and learners need, as well as a lack of connection
between WBLR design and educational standards. Also ITU Monitor (2009) reports that the se-
lection of digital learning resources is limited and that designing and developing WBLRs, which
provide added value in learning and teaching, is very demanding and time consuming. Neverthe-
less, some schools have made progress in the use of digital learning resources, but many of them
still have much to do to develop and use subject-specific digital learning resources.
Furthermore, the concept of learning environment (Wilson, 1998), has been used to describe
changes in the way teachers organize learning activities in classrooms around digital resources.
This indicates a change from learning environments, where the teacher and the textbook control
the learning process, towards constructivist learning environments, where the students them-
selves, not the teacher, control how to learn based on their needs. To realize their potential capa-
bilities, WBLRs need to be designed to support a learner-centered environment, where students
explore the content of the subject matter and enhance their learning through interactive, flexible,
differentiated, authentic, and motivating activities (John & Sutherland, 2009). Unfortunately,
much of the construction of WBLRs is carried out without a true understanding of pedagogy, is-
sues pertinent to learner control, and user involvement (Akpinar & Simsek, 2007; Farrell & Carr,
2007). As a result, current WBLRs provide little support to achieve a high level of flexibility, in-
teractivity, feedback, differentiation, and collaboration, diminishing the added value of WBLRs
(Liu & LaMont Johnson, 2005; Martinidale, Cates & Qian, 2005).
Summarizing, the literature review reveals a lack of approaches to the development of WBLRs in
school education. Existing approaches so far are limited (Memmel, Ras, Jantke & Yacci, 2007).
Likewise, the evaluation of WBLRs cannot be done in the same manner as traditional user-
centered testing, because WBLRs are embedded in a learning environment. This work sets out to
address these issues by proposing an approach to WBLRs for the identification of factors in rela-
tion to the development and evaluation of WBLRs.

Pedagogical Usability Principles

The importance of usability issues in education has been recognized, but these have not been suf-
ficiently researched (Hadjerrouit, 2005; Mayes & Fowler, 1999; Peterson, 2007; Simbulan 2007).
Nielsen (1993) proposed a definition that focuses on technical usability. This measures the extent
to which a software system is convenient, practicable, and usable for users. More specifically,
Nielsen’s criteria of Web usability (Nielsen, 2000) include content, page, and site design. Page
design is related to cross platform, speed of page access, page appearance and structure. Content
design depends on writing for scannability and media use. Site design is about linking and naviga-
tion. Technical usability is a self-evident requirement, but the impact of Nielsen’s definition is
limited when it comes to design WBLRs that are pedagogically usable. The usability concept
must be extended to capture issues that are fundamental to learning (Krauss & Ally, 2005). The
added value of WBLRs in terms of learning compared to teacher- and textbook-directed instruc-
tion lies in supporting the student to acquire knowledge through interactive, flexible, differenti-

118
 Hadjerrouit

ated, and motivating activities. For this reason, WBLR development needs to integrate issues per-
tinent to learning and pedagogical considerations (John & Sutherland, 2009).
The concept of pedagogical usability has been addressed by Nokelainen (2006), who defined a set
of ten criteria that can be applied to digital learning material: learner control, learning activity,
collaborative learning, goal orientation, applicability, added value, motivation, previous knowl-
edge, flexibility, and feedback. These criteria must be adapted to the specificities of WBLRs, be-
cause these systems cannot be measured in exactly the same terms as digital learning material.
Hence, Nokelainen’s criteria have been expanded to include the criteria of understandability,
time, multiple representation of information, autonomy, and variation. Interactivity is similar to
the feedback criterion. The criteria of applicability, previous knowledge, and added value are ex-
plained in greater details in Nokelainen (2006). These have not been explicitly considered in this
paper. As a result, the key criteria that influence the pedagogical usability of WBLRs are pro-
vided in Table 1.

Table 1: Pedagogical Usability Criteria

CRITERION CHARACTERISTICS
WBLRs should provide a well-structured description of the subject
Understandability
information using an understandable language.
Learner-control describes the student’s ability to control the order in
Learner-control
which they would like to perform activities.
Goal-orientation relates to the learning utility of WBLRs in terms of the
Goal-orientation
learning goals set by the teacher and the curriculum.
WBLRs must allow the student to learn the subject matter within a short,
Time
but acceptable, period of time.
Interactivity is supported through easy and user-friendly accessibility of
Interactivity
the subject information and task-based activities.
Multiple representation WBLRs should provide multiple representation of information using
of information various multimedia elements, e.g., text, graphics, images, and sounds.
The material provided by WBLRs should contain intrinsically motivating
Motivation
tasks and examples.
Differentiation involves fitting the subject information to the
Differentiation
characteristics of the students, taking into account their abilities.
WBLRs should provide different levels of difficulty and contain diverse
Flexibility
assignments and tasks that are tailored to the students.
Autonomy means that students are able to work on their own using
Autonomy
WBLRs, without being completely dependent on the teacher.
Students can work together to reach a common goal, giving them a sense
Collaboration
of how problem solving can be carried out in collaboration.
Students are able to use other learning resources in combination with
Variation
WBLRs, such as textbooks.

Technical and pedagogical usability are related to each other, and even congruent. They cannot be
considered as separate, disjointed activities (Tselios, Avouris, & Komis, 2008). Technical usabil-
ity involves techniques and methods for ensuring a trouble-free interaction with the WBLRs
while pedagogical usability aims to support the learning process. The goal is to minimize the
learners’ work resulting from the interaction with the WBLR in order to free more resources for
the learning process. Technical and pedagogical usability criteria need to be adapted to the char-
acteristics of school students (Nielsen, 2002, 2005) because there are important differences be-
tween them and adult users. School students like modest, but clear design. They like enjoyable
and visual appearance, online quizzes, and sound effects. They don’t like to read a lot on the

119
Developing Web-Based Learning Resources

Web. Interactive features work better for young users, because they let them do things rather than
read text. Young users also like Web sites that provide feedback.

User-Centered Development
WBLRs need to be technically and pedagogically usable before they can be accepted for use in
the classroom (Robles, 2007). Hence, the goal of the development approach is to help developers
to translate technical and pedagogical usability requirements into a system that supports effective
learning (Villani, 2007). To achieve this, the approach needs to incorporate three principles. First,
the approach is user-centered, because participation of users is given high priority. Then, the ap-
proach uses rapid prototyping in the design phase to produce a number of prototypes that can be
revised through user feedback (Farrell & Carr, 2007). Finally, the approach is incremental
throughout the whole process, because a number of revisions are necessary to improve the quality
of WBLRs through a continuous cycle of gradual refinement. The development approach consists
of four basic stages: analysis, design, implementation and testing, and evaluation (Figure 2).

Figure 2: WBLR development as continuous cycle of gradual refinement with four stages

Analysis Stage
Data collected during this initial stage serves as the foundation for the design of WBLRs. The
developers specify and delimit the problem to be solved, which has been identified by school
teachers. The developers also describe the objectives that need to be achieved with the WBLR.
Then, the developers need to get information from the school context in which the WBLR oper-
ates. Basically, the context can be determined through the identification of the elements that di-
rectly influence the WBLRs (Pal, 2003; Quiton, 2007; Strijker & Collis, 2007). The context is the
totality of relationships between the students and surrounding elements within a teaching and
learning situation. This concept of context is similar to Brousseau’s term of “milieu” (Brousseau,
1998), where learning is described as emerging from exchanges between the students and a “mi-
lieu” organized with teaching intentions. Accordingly, context is everything in the situation the
learners can act on. It can be divided into two categories: material and non-material context.
The material context includes a number of elements that are of importance for WBLR design.
First, the content of the subject taught in the classroom. Second, the curriculum and its objectives,
main subject areas, basic skills, including digital literacy skills, competence aims in the subject,
and subject assessment. Third, the computer infrastructure, software tools available, number and

120
 Hadjerrouit

place of computers, and the student/PC-ratio in the classroom. Finally, the textbooks used in the
classroom and other study material, such as teaching notes and electronic and written material.
The non-material context consists of the users and stakeholders involved in the development of
WBLR. First, school students are the main users of the WBLR. Developers need to obtain
information about their characteristics, such as age, knowledge level, gender, interests, needs,
motivations, and computer skills. Second, developers need information about school teachers and
their profile and pedagogies, ICT qualifications and attitudes towards ICT. Finally, it could be
important to collect information about the ICT policy of school leaders and decision makers
supporting ICT integration.
The context forms the very basis for the process underlying the design, implementation, use, and
evaluation of WBLRs. The context may change for the student when introducing a new element -
the WBLR, because learning and teaching processes are then mediated by the technology. Ac-
cording to Brousseau (1998), learning occurs by means of interaction between the student and the
“milieu”. Learning happens through adaptation of the student to the “milieu”. This view is consis-
tent with Piaget’s (1972) work that considers learning as an adaptation process to a “milieu”. The
WBLR can be considered as an element of the “milieu” or context, and as such, it becomes a
source of learning, by means of interaction with the student.

Design Stage
Design is the process of specifying how WBLRs are to be built and presented to the users. Basi-
cally, design is characterized by the process of transforming technical and pedagogical usability
criteria into a system that supports learning. The design stage consists of four steps. First, a meta-
phor is designed. This is followed by the user interface design. Then, the architecture of WBLRs
with the main components is determined. Finally, the design is revised through formative user
feedback. A cyclical process evolves between the four steps. The very basis of the design stage is
rapid prototyping, where a number of prototypes are developed, evaluated, and revised through
user feedback.

Metaphor design
The question is how to link technical and pedagogical usability requirements so that WBLRs con-
tribute to the learning process. According to Stenseth (1999), a good metaphor is essential to the
user interface in order for the learning to take place. It supports the cognitive model of the WBLR
that involves learning. In many ways, the metaphor is the personality of the WBLR. It permits a
user to relate the complexity of the Web to something previously experienced (Palmquist, 2001).
For this reason, the metaphor has huge consequences on the terminology, functionality, and
graphical design of WBLRs (Ratzan, 2000). The use of a metaphor is indeed an important issue
for any WBLR for many reasons. First, the metaphor gives a general setting where the learners
are placed. Second, the metaphor serves to explain the functionality of WBLRs. In addition, it
provides support for the learners to understand the content of the subject. The metaphor also sup-
ports the students’ learning, motivation, and communication with others. The metaphor situates
the learning with regard to place, time, and role, that is to say where and when the learning takes
place, and which role the users play. Examples of well-known metaphors are the dice metaphor in
the field of statistics and probability, and the basket metaphor in shopping.

User interface design

Based on an appropriate metaphor the design continues with the specification of the user interface
and associated functionality. To capture a number of features of the interface there is a need to

121
Developing Web-Based Learning Resources

describe it from the users’ point of view. A combination of three different techniques can be used
to express the functionality of WBLRs:
• Scenario to write down what the users do during the sessions with the WBLR
• Activity table to systematically organize the actions of the users in a table
• Graphical representation to show the main windows of the WBLR.
The first technique uses scenarios on the basis of natural language statements. Scenarios are used
to extract the verbs and to go on with the next two techniques. A scenario will never be complete
and cover all possible combinations of actions, but it should pick up typical work sequences
(Stenseth, 1999). Basically, scenarios are associated with the operations the users perform. There
are two types of scenarios: student scenarios and teacher scenarios. Students study the content of
the subject, undertake assignments and exercises, or evaluate the quality of WBLR. Teachers
provide new study material, update the existing ones, set up assignments, evaluate the students’
tasks, review learning activities, etc. The scenarios and the extracted verbs are the foundation for
distributing the responsibility for different actions between the main roles involved in WBLRs
(Stenseth, 1999). For WBLRs the typical roles are the student, the WBLR itself, and the teacher.
The second technique – the activity table - is used to organize the actions expressed by the verbs
that are extracted from the scenarios. Three roles and their actions are taken into account. Table 2
shows some examples of possible actions that can be taken by the student, the WBLR, and the
teacher.

Table 2: Activity table expressing the roles associated with the WBLR
STUDENT WBLR TEACHER
Start the WBLR Shows the result of a quiz Set up an assignment
Select a content page Show the results of student Set up new activities
evaluation
Solve a quiz Provide new study material
Look at related study material Evaluate the WBLR
Evaluate the WBLR

The third technique that helps to present the functionality to the users is the graphical interface.
The metaphor is the most important element of the interface. An example of a good metaphor is
the Web site that is organized around the metaphor of a tree house (Figure 3). Users navigate up
and down in the tree house to look at different children's books (Skaalid, 1999)

122
 Hadjerrouit

Figure 3: Example of a Web site that is organized around the metaphor of a tree house

Architecture design
This design phase is concerned with the architecture of the WBLR, which is usually hierarchical
with the top as the home page (Figure 4). The WBLR is divided into components that consist of
one or more pages. The navigation through the WBLR from one page to another is flexible with
many entries. A number of pages are interactive and designed with multimedia elements. Stu-
dents have the possibility to control the order of the activities they do. They may skip and revisit
pages. Control of sequence engages the students in flexible navigation paths and nonlinear learn-
ing. Flexible design is important, because some students learn the topics before doing exercises,
while other students prefer working with interactive tasks rather than reading theoretical topics.
Some students prefer WBLRs that open up for many entries, while other are satisfied with one or
two entries. Clearly, the use of WBLRs depends on the students’ needs.

Figure 4: WBLR architecture design

123
Developing Web-Based Learning Resources

User feedback
Early and frequent assessment and revision are important to ensure that the WBLRs meet the us-
ers’ goals and needs. User feedback can be considered as formative evaluation that occurs during
the design phase. Formative user feedback is used to guide the design of WBLRs, and ensure that
the users’ needs are kept in mind, and that the decisions made throughout the phases of analysis
and design are achieved.

Rapid prototyping
During the design phase, developers create a number of prototypes based on predetermined objec-
tives. Rapid prototyping creates an iterative and dynamic process where developers assess and
revise the prototypes to meet the users’ needs before the implementation of the final product takes
place in the classroom. Prototypes are developed and evaluated on the basis of user feedback,
both from students and teachers, to ensure that the objectives of WBLRs are taken into considera-
tion.

Implementation and Testing Stage

This phase is about the implementation of WBLRs and integration of multimedia elements using
programming languages and specialized authoring tools. During implementation, components
that realize some of the functionalities of the WBLRs may be reused. These may include reusable
units of software, such as video films that can be downloaded from the Web. WBLRs may also
reuse learning objects, such as lessons, exercises, quizzes, learning activities, and written texts
from past or similar projects. Before installation in the classroom for use, WBLRs must be tested
systematically. Testing is the process of exercising WBLRs with the intent of finding and correct-
ing various errors, such as typographical mistakes, errors in content and graphical representations,
cross referencing and navigation errors. This phase also includes content review, browser and
operating system compatibility, performance testing, etc.

Evaluation Stage
After the use of the WBLRs in the classroom, summative evaluation takes place in order to assess
the quality of the final product and to review the overall development process. The goal of the
evaluation is not simply to prove the technical effectiveness of the WBLRs, but rather to assess
the pedagogical value and learning effect of WBLRs. Summative evaluation contributes to the
formative evaluation that occurs throughout the design phase by means of rapid prototyping and
user feedback. A number of evaluation instruments can be used to assess the value of WBLRs,
such as survey questionnaires, individual and group interviews, usability testing questionnaires,
and video-taped observation sessions. These are relevant instruments to obtain data on what both
students and teachers think about WBLRs. Finally, experts’ opinions in educational research and
developers’ views are good instruments to elicit the value of WBLRs. The evaluation of the final
product is described in greater details in another paper (Hadjerrouit, 2010).

Application Example
To illustrate the use of the development approach in school education an application example is
described.

Developers of WBLRs
The WBLRs were developed by three teams of trainee teachers from the University of Agder in
collaboration with the university teacher who provided academic supervision, on the one hand,
and school teachers who informed the trainee teachers about curricular and pedagogical issues, on

124
 Hadjerrouit

the other hand. Collaboration occurred by means of e-mail, phone, and meetings. The developers
did not have any experience in developing WBLRs using a user-centered development approach,
but, they were able to develop simple, mostly static WBLRs for subject teaching. They also had
basic skills and knowledge in using ICT for learning.

WBLRs
The objectives that needed to be achieved with the WBLRs were aligned with the National Cur-
riculum for schools and the competence aims that are specified for each subject. The way of using
WBLRs in the classroom was defined by the school teachers, who considered WBLRs as sup-
plementary resources in addition to textbooks. The main reason for using WBLRs was the in-
creased focus on digital literacy skills in the curriculum, which now defines digital literacy as a
key area of competence. The situation, in which the WBLRs were developed, is listed in Table 3.
Each row refers to the class, the school subject, the grade, and the number of participating stu-
dents. The school subjects were taught in three different classes in a middle school.

Table 3: School subjects and associated WBLRs

CLASS SCHOOL SUBJECT GRADE NO. STUDENT
Class 1 Language and culture 10. grade 18
Class 2 Water – A vital resource 8. grade 24
Class 3 Volcanoes, tsunamis, and earthquakes 8.grade 23

The WBLR in class 1 was designed to support students in the 10th grade to explore the Norwegian
language from an historical, cultural, and linguistic perspective. The WBLR in class 2 was con-
ceived to help students in the 8th grade to explore water as a vital resource from a natural science,
societal, and political perspective. Finally, the objective of the WBLR in class 3 was designed to
help students in the 8th grade to explore the characteristics of volcanoes, tsunamis, and earth-
quakes in different regions of the world.
The technologies used for implementing WBLRs were the scripting languages JavaScript and
PHP, Macromedia Flash, HTML/CSS, and diverse tools for recording video film, sounds, and
other multimedia elements. WBLRs were tested in a heterogeneous computing environment that
includes multi-platforms, multi-browsers, and multimedia support.

Users
The users of the WBLRs were 65 school students between 14-16 years from three different
classes. The students had little experience in using WBLRs in their learning of school subjects.
In addition, three school teachers from three different classes were involved in the development
process. The teachers had solid knowledge in pedagogy, but little experience in using WBLRs.

Developers’ Experiences with the User-Centered Approach

Data for describing the developers’ views came from the analysis of project reports, informal dis-
cussions, and observations of the development process. Although the developers were globally
satisfied with the WBLRs both from the technical and pedagogical point of view, they were con-
fronted with a number of problems associated with pedagogical issues, design considerations, as
well as organizational and institutional barriers within the school.
The analysis stage was characterized by the difficulty of specifying the problem requirement, de-
limitating the problem scope, and understanding the users’ needs. This difficulty was partly re-

125
Developing Web-Based Learning Resources

solved in collaboration with school teachers who provided information about the problem state-
ment and competence aims that are specified for each subject in the curriculum. School teachers
also provided information about the students’ characteristics and profiles. Furthermore, the de-
velopers were confronted with the task of finding reliable information about the content of the
WBLRs. Looking at the repository of past project applications available online was helpful, be-
cause some elements of the applications could be adapted and reused with slight modifications to
meet the requirements of the new projects. Likewise, the study of textbooks and study material
used in schools provided support for content design.
Despite these difficulties, rapid prototyping helped the developers to quickly develop a number of
prototypes that could be assessed in collaboration with the instructor. In addition, the assessment
of the prototypes was done in collaboration with school teachers, who provided information about
pedagogical considerations, but in lesser degree than originally expected. Unfortunately, student
feedback was insignificant, and, as a result, revisions of prototypes could not be carried out with
representative users. Furthermore, one of the most difficult problems was the design of an appro-
priate metaphor that is technically and pedagogically usable. Also the question of page size and
content of the home page created some difficulties. The design was improved through a cyclical
process of revisions of the user interface, and overall architecture of the WBLRs.
Furthermore, the developers reported that the implementation process was very demanding in
terms of efforts and time, mostly because of the high workload required to design a number of
multimedia elements, such as video films, images, sound files, and animations. Also the scripting
language PHP for the programming of quizzes and evaluation forms was a difficult task for two
project teams, because of the lack of prerequisite knowledge in the language being used.
The developers indicated that the WBLRs need to be improved through a better understanding of
students’ learning preferences and their sequencing, for example their preferences for animations,
images, and recited text. One project team expressed this concern as follows: “Clearly, students
liked quizzes, but as the resource is built, they need to go through all the topics before they can
begin with the quizzes. This could be too much information at once. Quizzes would be more useful
if they are linked to each subtopic”. Likewise, the second project team believed that the students
need more quizzes and images that illustrate the study material. The third team also reflected
upon what needs to be improved, such as the lack of recited text instead of written material.
Clearly, the developers were aware of the fact that some elements of the WBLRs need to be im-
proved to meet the users’ needs.
Finally, the evaluation of the WBLRs was conducted after their use in the classrooms by means
of electronic survey questionnaires. The questionnaires and the underlying criteria were designed
in collaboration with the instructor. Also here, two project teams were confronted with problems
associated with time delay, the school time frame, and heavy teachers’ workload. However, this
phase did not create insurmountable difficulties that could hinder the evaluation phase.

Users’ Perceptions of WBLRs

School students and teachers were asked to describe their perceptions of the final products after
their use in their respective classes. This was done by means of student and teacher survey ques-
tionnaires. These focused on what the students and teachers had to say about the technical and
pedagogical usability of WBLRs. In addition, both students and teachers were asked to respond to
two open-ended questions and comment what they liked and disliked about the WBLRs, and what
should be improved. The following sections give a brief summary of the results. The evaluation
framework and results were described in greater details in another paper (Hadjerrouit, 2010).

126
 Hadjerrouit

School students’ perceptions of WBLRs

First, the results show that most students unanimously agreed that the technical usability of
WBLRs was well-designed. The criteria for judging technical usability were page, content, and
site design. They also indicated that the level of pedagogical usability was high with regard to all
criteria listed in Table 1, except for collaboration and variation. More specifically, most students
indicated their preferences for interactive animations, audio recording, video films, graphics, and
recited text over written material. Furthermore, most students think that they were motivated
when they used WBLRs. Moreover, they believed that the WBLRs showed a high degree of dif-
ferentiation. Finally, most students reported that they did not collaborate with their fellow stu-
dents, nor did they ask for teacher help or use textbooks in combination with the WBLRs.
These results were confirmed by the comments the students made about the WBLRs. Most stu-
dents from class 1 (Language and culture) liked the content provided by the WBLR, especially
the combination of written and recited text with audio recording. Most students preferred the
WBLR more than textbooks. Although the majority of the students were satisfied with the
WBLR, some of them expected more interactive animations, audio recording of text, and video
films. Students from the second class (Water – a vital resource) pointed out that they liked very
much this kind of WBLR, especially sound recording, quizzes, and the content. Most students did
not think that the WBLR should be improved substantially, but some expected more interactive
exercises. Finally, the students from class 3 (Volcanoes, tsunamis, and earthquakes) liked very
much the WBLR, especially the content, animations, and quizzes. However, most students think
that there is a need for more interactive animations and recited text with audio recording. Some
students expressed some concerns about the technical design of the video films.

School teachers’ perceptions of WBLRs

The results show that all teachers unanimously believed that the criteria underlying the technical
usability were well-designed and implemented. Teachers also agreed that the dimensions of the
pedagogical usability were globally achieved. Moreover, teachers reported that the degree of stu-
dents’ independence from teacher assistance was high, in stark contrast to the low level of col-
laboration with fellow students and the low degree of variation of learning resources.
The teacher from the project “Language and culture” reported that the audio recording was fine.
The teacher also considered as a positive attitude towards the learning process that students can
work with the subject matter individually and have the possibility to acquire new knowledge. In
addition, the teacher thinks that the WBLR realizes differentiated learning. Likewise, the teacher
from the project “Water–a vital resource” indicated that recited text was fine and the degree of
difficulty of the content was varied. However, the teacher believed that the WBLR will never
cover the entire scale of the students’ knowledge through quizzes; other task-based activities must
be offered to satisfy the students’ needs. Finally, the teacher from the project “Volcanoes, tsuna-
mis, and earthquakes” believed that the WBLR is characterized by a good layout, but there was a
lack of information about the composition of the globe. Finally, the teacher indicated that the
WBLR could be improved by showing a cut through the globe and the continuous and dynamic
flow of magma. It is also important to show examples of mountains that are formed by collisions,
for example the Himalayas.

Relationships between Developers’ and Users’ Views

A careful analysis of developers’ and school students’ views shows that their perceptions of tech-
nical usability were globally similar as both believed that the WBLRs were technically well-
designed. Furthermore, there is a high degree of similarity regarding pedagogical usability, al-
though student feedback during the design phase was insignificant and, as a result, revisions of
the prototypes were not carried out with representative users. It appears that the analysis and re-

127
Developing Web-Based Learning Resources

use of past project applications helped the developers to obtain some information about the stu-
dents’ learning preferences. Likewise, a careful analysis of the developers’ and school teachers’
views shows that they globally agreed that multimedia and interactivity are important in WBLR
design, despite the fact that teacher feedback was not as good as expected. While both agreed that
the provided learning activities in form of quizzes, recited text, and animations were globally sat-
isfying, both believed that the WBLRs need to be improved to better achieve the learning goals
and anticipate the design of WBLRs in a more satisfactory way. Hence, there is a need for more
advanced and, preferably, more differentiated activities.

Evaluation and Discussion

From a theoretical point of view, the user-centered approach is potentially powerful for develop-
ing WBLRs. From a practical point of view, trainee teachers must possess sufficient skills in or-
der to carry out the tasks required for developing WBLRs in authentic school environments, de-
spite the fact that they were not educated as software experts. On the other hand, their pedagogi-
cal background in teacher education is clearly an advantage when it comes to pedagogical design.
They also need to understand the limitations and potential capabilities of WBLRs, as well as the
ways schools introduce WBLRs. In addition, trainee teachers are expected to do administrative
work, organize meetings with school teachers and students, design procedures for using WBLRs
in the classroom, and eventually improve and maintain the WBLRs after their use in the class-
room (Combes & Valli, 2007). In addition, to deal with practical challenges, trainee teachers must
be in a position to negotiate the timetable with school teachers and deal with organizational con-
straints and other obstacles, otherwise WBLRs cannot have the desired effect. Given these con-
siderations, it appears that the development approach may be inhibited or hindered by external
factors that are outside the developers’ control, such as the school timetable that is tightly regi-
mented.
Given these constraints, the development approach can be evaluated in a variety of ways (Stand-
ing, 2002). First, it can be evaluated according to a framework of requirements to determine if
these are met by the approach. The benefit of this method is that it can be carried out by the de-
velopers themselves, since they have sufficient knowledge background in WBLRs. The weakness
is that all problems are unlikely to be identified because trainee teachers are not software experts.
Second, focus groups are another method of evaluating the development approach. This method
has the advantage of getting input from a range of practitioners and experts in the field of
WBLRs. The limitation is that all issues and problems may not be identified until the approach is
used in practice. In addition, school teachers have rarely the time and all the skills needed to eva-
luate WBLRs. They have little control over technical issues, unless the school uses professional
developers and usability specialists to assist them. Finally, the approach can be evaluated by
adopting it as the development approach on a project as a form of action research or design-based
research (Barab & Squire, 2004; Baskerville, 1999). According to Standing (2002), this is an
ideal approach for evaluating a development approach.

Evaluation Requirements
This approach was evaluated according to a framework of requirements that is based on the work
of Montilva, Sandia, and Barrios (2002). This consists of four general evaluation principles that
may be applied to any software development approach. The first principle is related to user in-
volvement and usability properties of the approach. The second principle is concerned with the
process model. The third principle relies on the idea that any development approach must have a
clear specified domain, and the fourth principle is related to the product model of the approach.
As a result, the evaluation framework consists of a set of four requirement principles: usability
properties and user involvement, process model, applicability domain, and product model. Each

128
 Hadjerrouit

principle can be described with a set of attributes. The usability and user involvement principle
includes the applicability and completeness of the development approach, usability properties,
and user involvement. The attributes of the process model principle are related to the process
model of the approach, management processes, and development processes. The set of attributes
of the applicability domain are the scope of the approach and the pedagogical domain of WBLRs.
These attributes include the mode of teaching, the school subject type, and the educational level
or grade. The attributes of the product model principle are concerned with the product model of
the approach, modeling language and notation, as well as the concepts underlying the model.

Usability and user involvement

From the theoretical point of view, the development approach covers the whole software lifecycle
from analysis to testing, including the evaluation stage. Likewise, the iterative character of
WBLRs is explicitly emphasized through a series of revisions. Furthermore, the approach shows
the stages of the development process and how to perform them, especially the design stage
through rapid prototyping. It uses standard techniques, and it is easy to use and flexible enough to
be adapted to a variety of educational situations. Finally, the approach involves the users in the
development process. User involvement is important to ensure that the pedagogical perspective is
taken into consideration.
From the practical point of view, the analysis of the project reports indicated that the developers
did not strongly emphasize pedagogical usability in the design phase, partly due to the difficulty
of specifying an appropriate metaphor that is technically and pedagogically usable. This resulted
in insufficient pedagogical design that impacted the whole development process, which in turn
required more revisions than originally anticipated. According to the approach’s main principles,
the use of pedagogical criteria should be considered earlier in the design process, and the devel-
opers should begin attempting to apply the criteria already in the design process. Moving the em-
phasis on pedagogy to the beginning of the development process and providing the developers
with a set of principles and heuristics can have a positive impact both on the development process
and the pedagogical quality of the final product. Furthermore, emphasizing pedagogical usability
in the earlier phases of the development process helps to achieve an alignment with school values
and learning scenarios favored by teachers and learners. However, this is a difficult task that can-
not be done without preparation and pre-work. One difficult was the specification of an appropri-
ate and consistent metaphor in the very beginning of the development process.
In addition, involving school teachers in the development process is an important principle of the
approach from a pedagogical point of view. Unfortunately, feedback from school teachers hap-
pened only a few times during the whole development process due to a number of factors, such as
the school time frame and heavy work load. Nevertheless, teacher feedback at any point of the
process is useful to improve the pedagogical quality of WBLRs. It helps to return to the main
stages of the process to improve and revise a number of elements, such as component size, navi-
gation, and differentiation of knowledge, or to include motivating quizzes and task-based activi-
ties, etc. Rapid prototyping allows these revisions to be made to a certain degree by means of ear-
ly and frequent teacher feedback before the installation of the WBLRs in the classroom.
Finally, rapid prototyping also stresses the importance of a thorough analysis of students’ needs,
and early and frequent assessment of the WBLRs to meet the learning objectives (Krauss & Ally,
2005). This means that representative students should be involved throughout the design phase to
ensure that most of their needs are met. Involving students during the design stage has many ad-
vantages. The most important drawbacks of WBLRs can be detected and corrected before they
are implemented and delivered. In addition, involving students from the very beginning ensures
that relevant pedagogical perspectives and students’ learning preferences are taken into consid-
eration. Unfortunately, frequent learner involvement in the design phase was even more difficult

129
Developing Web-Based Learning Resources

to achieve than teacher feedback, because of the school time frame and institutional constraints,
which clearly limited learner feedback.

Process model
The process model of the development approach provides insight into the specificity of analyzing,
designing, implementing, testing, and evaluating WBLRs in school education. However, WBLR
development differs from other approaches in a number of features. Approaches that use conven-
tional software development approaches (Pressman, 2000), or extensions of these approaches to
Web-based learning (Horton & Lynch, 1999), to address the whole process are not flexible
enough, or too complicated, to be applied to WBLRs. Domain-specific approaches that are exclu-
sively devoted to Web-based development (Balasubramaniam, Pries-Heje, & Baskerville, 2003;
Murugesan & Ginige, 2001) do not explicitly address the specific characteristics that are proper
to WBLRs, such as pedagogical issues, user-centered design, evolution and change. The approach
described in Montilva, Sandia, and Barrios (2002) takes into consideration instructional aspects,
such as education level, modality, and course type, but not pedagogical and learning dimensions
in school education.
In addition to the specificities of the approach, a number of practical issues need to be addressed
in order to improve the overall process model. First, the approach lacks a project management
process. This is concerned with project planning activities that are required to manage the devel-
opment process. The objective of project planning is to provide a framework that enables the pro-
ject manager to make estimates of resources, costs, risks, and schedule (Pressman, 2000). These
estimates are made within a limited time frame at the beginning of the project and should be up-
dated as the project progresses. But given the specificities of WBLRs, a strict management proc-
ess model may not be necessary to perform the activities connected to the development process
due to a number of influencing factors, such as the users’ needs that are difficult to assess, institu-
tional and organizational barriers, time delay, school time frame, and teachers’ heavy work load
(Pan, Gay, & Saylor, 2007). Hence, the development of WBLRs requires a slightly different
process than originally anticipated. It appears that constructing WBLRs is a product of a creative
act of development with a number of unexpected events that occur during the development proc-
ess, and not a result of repetitive act of manufacturing. Clearly, the development of WBLRs is not
always a straightforward task, following a clear procedure. For this reason a rigorous, strict, and
systematic approach is not the best solution for developing WBLRs in order to avoid faulty solu-
tions and poor design quality. Clearly, the approach must be flexible and cyclical in order to deal
with a number of unexpected events and influencing factors.
Finally, it is not necessary to develop WBLRs from the ground up, since reuse is an essential
element of the approach (Johnson, & Hall, 2007). The reuse philosophy of the approach relies on
the basic idea that WBLR project activities are similar. Hence, elements of past projects may be
adapted and reused to meet the requirements of new projects. However, students need to acquire
some experience in analogical reasoning in order to achieve effective reuse, such as recognizing
structural similarities and differences between past and new projects (Duit, Roth, Komorek &
Wilbers, 2001). Similar components may be navigation menus, page structure, quizzes, and
evaluation forms. In addition, reusability helps developers to reduce the amount of implementa-
tion work. However, this is a difficult task. The implementation phase was indeed very demand-
ing, because some trainee teachers focused on coding from the ground up, instead of reusing pre-
vious code from past projects. Clearly, developers must be encouraged to take an active role in
constructing their own understanding of past project applications in order to reuse some compo-
nents in their own projects.

130
 Hadjerrouit

Application domain
The application domain and the coverage of the approach include both software processes and
pedagogical issues. Particularly important for WBLRs is the consideration of the pedagogical
domain, which is explicitly specified with the concept of “milieu”, where learning is described as
emerging from exchanges between the students and a “milieu”. In addition, the approach provides
an explicit way for developing WBLRs in school education. It is applicable to a wide range of
situations, where schools are expecting to gain learning benefits in investing in WBLRs. It is flex-
ible enough to be adapted to the specialized conditions of school institutions. More specifically,
its application area includes different levels of school education ranging from primary to upper
secondary schools. It can be classroom and distance-based or a combination of them. Further, a
variety of WBLRs for school subjects may be developed using the development approach.

Product model
The product model is concerned with the explicit representation of the WBLRs. This is described
by the characteristics of the teaching subject, the educational level and grade, the number of the
students, and the technology being used. The approach clearly defines the properties of the
WBLR by describing its features and dimensions, both technical and pedagogical concepts. Fur-
thermore, the approach uses scenarios with textual notations to write down what the users do dur-
ing the sessions with the WBLR, activity tables to systematically organize the actions of the users
in a table, and graphical representation to show the main windows of the WBLR. Finally, the
product model is described and evaluated from different perspectives: learner, teacher, and devel-
oper. Evaluation methods by means of survey questionnaires include both technical and peda-
gogical usability criteria to ensure that WBLRs are evaluated consistently.

Limitations
The primary contribution of this paper is to use a user-centered development approach to produce
pedagogically usable WBLRs and to show how these resources could be designed and used to
enhance the students’ learning. However, a number of limitations need to be considered in inter-
preting the evaluation results. Firstly, the duration of the development process was too short to
develop advanced WBLRs. To assess the value of the approach over a long period of time it may
be necessary to improve the quality of the WBLRs on the basis of the evaluation results. Refine-
ments of the approach can be made through successive cycles of revisions, where the shortcom-
ings of each cycle are identified, re-designed, re-implemented, and re-evaluated. Secondly, the
evaluation results cannot be generalized, because of various factors, such as a single experiment
of limited duration, the small sample size (N=65), the small number of classes (N=3), three
school teachers, and three different topics. Nevertheless, new examples of applications and case
studies in school education are warranted to generalize the results of the present work.

Conclusions and Future Research Work

This work proposes a user-centered approach to the development process of WBLRs in school
education. The approach was evaluated from three different perspectives: trainee teachers, school
teachers, and school students. The approach needs to be further developed through continuous
cycles of design, implementations, and evaluations in various school contexts. The approach pro-
vides re-entry points into the analysis, design, implementation, and evaluation stages to allow for
continuous improvements. For example, developers may return to the design phase to revise se-
lected objects that are important for the learning process. They may also return to the analysis
stage to re-examine the context of use, influencing factors, or learning goals. Results also stress
the importance of a thorough user-needs analysis, as well as early and frequent assessment of pro-
totypes with the users. The pedagogical use of WBLRs in school environments and reuse of

131
Developing Web-Based Learning Resources

learning objects is another potential area for research. Through the continuous cycle of develop-
ment, the author hopes to explore WBLRs in more details and depth in order to gain more theo-
retical and practical insights.

References
Akpinar, Y., & Simsek, H. (2007). Teachers’ learning object development: A case study in K-12 setting.
Interdisciplinary Journal of E-Learning and Learning Objects, 3, 197-217. Retrieved February 5,
2009 from: http://www.ijello.org/Volume3/IJKLOv3p197-217Akpinar.pdf
Balasubramaniam, R., Pries-Heje, J., & Baskerville, R. (2003). Internet software engineering: A different
class of processes. Annals of Software Engineering, 14, 169-195.
Barab, S., & Squire, K. (2004). Design-based research: Putting a stake in the ground. The Journal of the
Learning Sciences, 13(1), 1-14.
Baskerville, R. L. (1999) Investigating information systems with action research. Communications of the
Association for Information Systems 2(19), 1-31.
Brinck, T., Gergle, D., & Wood, S.D. (2002). Usability for the web: Designing web sites that work. San Fran-
cisco, CA: Morgan Kaufman.
Brousseau, G. (1998). Theory of didactical situations. London: Kluwer Academic publishers.
Combes, B., & Valli, R. (2007). The future of learning objects in educational settings. In K. Harman & A.
Koohang (Eds.), Learning objects: Applications, implications, & future directions (pp. 423-461). Santa
Rosa, CA: Informing Science Press.
Duit, R., Roth, W.-M., Komorek, M., & Wilbers, J. (2001). Fostering conceptual change by analogies –
between Scylla and Charybdi. Learning and Instruction, 11, 283–303.
Erstad, O. (2006). A new direction? Digital literacy, student participation and curriculum reform in Nor-
way. Education and Information Technologies 11, 415-429.
Farrell, K., & Carr, A. E. (2007). A blended model of instructional design for learning objects. In A. Koo-
hang & K. Harman (Eds.), Learning objects and instructional design (pp. 359-405). Santa Rosa, CA:
Informing Science Press.
Govindasamy, T. (2002). Successful implementation of e-learning: Pedagogical considerations. The Inter-
net and Higher Education, 4, 287-299.
Hadjerrouit, S. (2005). Web-based educational software in computer science: Technical and pedagogical
usability. Proceedings of ED-MEDIA 2005, 1139-1144.
Hadjerrouit, S. (2010). A conceptual framework for using and evaluating Web-based learning resources in
school education. Journal of Information Technology Education, 9, 53-79. Available at
http://www.jite.org/documents/Vol9/JITEv9p053-079Hadjerrouit743.pdf
Hamid, A. A. (2002). E-Learning: Is it the “e” or the learning that matters? The Internet and Higher Educa-
tion, 4, 311-316.
Horton, S., & Lynch, P.J. (1999). Web style guide: Basic design principles for creating Web sites. London:
Yale University Press.
Ingram, A. (2003). Usability of alternative Web course structures. Computer in the Schools, 19(3), 33-47.
ITU Monitor. (2009). The digital state of affairs in Norwegian schools. Retrieved October 17, 2009 from:
http://www.itu.no/ITU+Monitor.9UFRDSXH.ips
Johnson, K., & Hall, T. (2007), Granularity, reusability and learning objects. In A. Koohang & K. Hartman
(Eds.), Learning objects: Theory, praxis, issues, and trends (pp. 181-207). Santa Rosa, CA: Informing
Science Press.

132
 Hadjerrouit

John, P., & Sutherland, R. (2009). Teaching and learning with ICT: New technology, new pedagogy? Edu-
cation, Communication & Information, 4(1), 101-107.
Kay, R., & Knaak, L. (2005). Developing learning objects for secondary school students: A multi-
component model. Interdisciplinary Journal of E-Learning and Learning Objects, 1, 229-254. Re-
trieved February 5, 2010 from: http://www.ijello.org/Volume1/v1p229-254Kay_Knaack.pdf
Kay, R., & Knaak, L. (2008). Investigating the use of learning objects for secondary school mathematics.
Interdisciplinary Journal of E-Learning and Learning Objects, 4, 269-289. Retrieved February 5, 2010
from: http://www.ijello.org/Volume4/IJELLOv4p269-289Kay.pdf
Kay, R., Knaak, L., & Petrarca, D. (2009). Exploring teachers perceptions of web-based learning tools.
Interdisciplinary Journal of E-Learning and Learning Objects, 5, 27-50. Retrieved October 22, 2009
from: http://www.ijello.org/Volume5/IJELLOv5p027-050Kay649.pdf
Krauss, F., & Ally, M. (2005). A study of the design and evaluation of a learning object and implications
for content development. Interdisciplinary Journal of Knowledge and Learning Objects, 1, 1-18. Re-
trieved September 22, 2009 from: http://www.ijello.org/Volume1/v1p001-022Krauss.pdf
Kukulska-Hulme, A., & Shield, L. (2004). Usability and pedagogical design: Are language learning web
sites special? Proceedings of ED-MEDIA 2004, 4235-4242.
Laurillard, D. (2002). Rethinking university teaching: A conversational framework for the effective use of
learning technologies (2nd ed.). London: Routledge.
Leacock, T. L., & Nesbit, J. C. (2007). A framework for the quality of multimedia resources. Educational
Technology & Society, 10(2), 44-59.
Liu, L., & LaMont Johnson, D. (2005). Web-based resources and applications. Computer in the Schools,
21(3), 131-147.
Martinidale, T., Cates, W. M., & Qian, Y. (2005). Analysis of recognized web-based educational resources.
Computers in the Schools, 21(3), 101-117.
Mayes, J. T., & Fowler, C. J. (1999). Learning technology and usability: A framework for understanding
courseware. Interacting with Computers, 11(5), 485-497.
Memmel, M., Ras, E., Jantke, K. P., & Yacci, M. (2007). Approaches to learning object oriented instruc-
tional design. In K. Hartman & A. Koohang (Eds.), Learning objects and instructional design (pp.
281-236). Santa Rosa, CA: Informing Science Press.
Montilva, J. A., Sandia, B., & Barrios, J. (2002). Developing instructional web sites – A software engineer-
ing approach. Education and Information Technologies, 7(3), 201-224.
Murugesan, S., & Ginige, A. (2001). The essence of web engineering—Managing the diversity and com-
plexity of web application development. IEEE Multimedia, 8(2), 22-25.
Nam, C. S., & Smith-Jackson, T. L. (2007). Web-based learning environment: A theory-based design proc-
ess for development and evaluation. Journal of Information Technology Education, 6, 23- 44. Re-
trieved from: http://www.jite.org/documents/Vol6/JITEv6p023-043Nam145.pdf
Nielsen, J. (1993). Usability engineering. Boston, MA: Academic Press.
Nielsen, J. (2000). Designing web usability: The practice of simplicity. Indianapolis: New Riders.
Nielsen, J. (2002). Web usability for children. Retrieved September 20, 2009 from:
http://www.useit.com/alertbox/children.html
Nielsen, J. (2005). Usability of websites for teenagers. Retrieved September 20, 2009 from:
http://www.useit.com/alertbox/teenagers.html
Nokelainen, P. (2006). An empirical assessment of pedagogical usability criteria for digital learning mate-
rial with elementary school students. Educational Technology & Society, 9(2), 178-197.

133
Developing Web-Based Learning Resources

Pal, C. (2003). Managing evolution in web-based teaching and learning environments. Computers & Edu-
cation, 40(2), 99-114.
Palmquist, R. A. (2001). Cognitive style and users’ metaphors for the web: An exploratory study. The
Journal of Academic Librarianship, 27(1), 24-32.
Pan, B., Gay, G., & Saylor, J. M. (2007). Learning objects in classrooms. In K. Harman & A. Koohang
(Eds.), Learning Objects: Applications, implications, & future directions (pp. 63-88). Santa Rosa, CA.
Informing Science Press.
Peterson, D. (2007). Usability theory, practice and evaluation for learning objects. In K. Harman & A.
Koohang, Learning objects: Applications, implications, & future direction (pp. 337-370). Santa Rosa,
CA: Informing Science Press.
Piaget, J. (1972). The psychology of the child. New York: Basic Books.
Pressman, P. (2000). Software engineering: A practitioner’s approach (5th ed.). London: McGraw-Hill.
Quiton, S. R. (2007). Contextualization of learning objects to derive meaning. In K. Harman & A. Koohang
(Eds.), Learning objects: Applications, implications, & future directions (pp. 113-179). Santa Rosa,
CA: Informing Science Press.
Ratzan, L. (2000). Making sense of the web: A metaphorical approach. Information Research, 6(1). Re-
trieved September 25, 2009 from: http://informationr.net/ir/6-1/paper85.html
Robles, M. M. (2007). Applying instructional design theory when using learning objects. In A. Koohang &
K. Harman (Eds.), Learning objects and instructional design (pp. 407-436). Santa Rosa, CA: Inform-
ing Science Press.
Simbulan, M. S. (2007). Learning objects’ user interface. In K. Harman & A. Koohang (Eds.), Learning
objects: Applications, implications, & future directions (pp. 259-336). Santa Rosa, CA: Informing Sci-
ence Press.
Skaalid, B. (1999). Metaphor of tree house. Retrieved September 20, 2009 from:
http://www.usask.ca/education/coursework/skaalid/site/metaphor/klutz.htm
Standing, C. (2002). Methodologies for developing web applications. Information and Software Technol-
ogy 4, 151-159
Stenseth, B. (1999). User centered program design. Retrieved July 5, 2009 from:
http://www.ia.hiof.no/~borres/marketmet/
Strijker, A., & Collis, B. (2007). The influence of context on the future of learning objects. In A. Koohang
& K. Hartman (Eds.), Learning objects: Theory, praxis, issues, and trends (pp. 83-112). Santa Rosa,
CA: Informing Science Press.
Tselios, N., Avouris, N., & Komis, V. (2008). The effective combination of hybrid usability methods in
evaluating educational applications of ICT: Issues and challenges. Education and Information Tech-
nologies, 13, 55-76.
Villani, S. L. (2007). Learning objects and instructional design: Theories, methods and tools useful in creat-
ing effective e-learning environments. In K. Harman & A. Koohang (Eds.), Learning objects: Applica-
tions, implications, & future directions (pp. 371-416). Santa Rosa, CA: Informing Science Press.
Wilson, B. G. (Ed.) (1998). Constructivist learning environments: Case studies in instructional design.
New Jersey: Educational Technology Publications Englewood Cliffs.
Winograd, T. (1996). Bringing design to software. New York: Addison-Wesley.

134
 Hadjerrouit

Biography
Said Hadjerrouit received MS and PhD degrees in Software Engi-
neering and Artificial Intelligence from the Technical University of
Berlin (Germany), in 1985 and 1992, respectively. He joined Univer-
sity of Agder, Kristiansand (Norway) in 1991. He is currently an Asso-
ciate Professor of Computer Science at the Faculty of Technology and
Sciences. He has been in the teaching profession for 28 years. He has
extensive experience in teaching object-oriented programming, Web
engineering, software development, databases, didactics of informatics,
ICT in mathematics education, ICT and learning. His research interests
include object-oriented software development, software engineering
education, didactics of informatics, ICT in mathematics education, e-Learning, Web-based learn-
ing resources, Web 2.0 tools and social software. Hadjerrouit has published over 80 papers in
international journals and conference proceedings. Web: http://home.uia.no/saidh/

135