Odzaly2018 Article AgileRiskManagementUsingSoftwa PDF

J Ambient Intell Human Comput (2018) 9:823–841
DOI 10.1007/s12652-017-0488-2
ORIGINAL RESEARCH
Agile risk management using software agents

Edzreena Edza Odzaly1,2 · Des Greer1 · Darryl Stewart1
Received: 18 January 2017 / Accepted: 5 April 2017 / Published online: 2 May 2017
© The Author(s) 2017. This article is an open access publication
Abstract Risk management is an important process in 1 Introduction

Software Engineering. However, it can be perceived as
somewhat contrary to the more lightweight processes used Risk management is recognized as a key process area in
in Agile methods. Thus an appropriate and realistic risk software development. Most risk management literature
management model is required as well as tool support that relates to heavyweight plan-driven processes and typi-
minimizes human effort. We propose the use of software cally assumes that, for example, requirements have been
agents to carry out risk management tasks and make use of agreed and signed off in advance of development. On the
the data collected from the project environment to detect other hand, Agile Software Development uses an iterative
risks. This paper describes the underlying risk management approach to software construction, aimed at reducing devel-
model in an Agile risk tool where software agents are used opment time, prioritising value, while improving software
to support identification, assessment and monitoring of quality and inherently reducing risk (Cockburn and High-
risk. It demonstrates the interaction between agents, agents’ smith 2001). This paper intends to demonstrate the idea
compliance with designated rules and how agents can react of software agents to help manage risks in project devel-
to changes in project environment data. The results, dem- opment. This is achieved by using software agents to carry
onstrated using case studies, show that agents are of use for out risk identification, risk assessment and risk monitoring,
detecting risk and reacting dynamically to changes in pro- the agents making use of data collected from the project
ject environment thus, help to minimize the human effort in environment. In the next section, we have highlighted the
managing risk. issues identified in risk management for an agile environ-
ment which further will be used as an input to the tool. In
Keywords Software risk management · Agile risks · the approach used, the project manager has to define these
Agile projects · Software agents elements: project goals, problem scenarios, consequences,
risk indicators, project environment data as well as speci-
fying risk rules using a predefined ‘Rule template’. Next,
the proposed Agile risk tool (ART) model is discussed,
focusing on the development of the tool. This shows how
the risk management activities are decomposed into agents,
* Edzreena Edza Odzaly
eodzaly01@qub.ac.uk
as well as how the interaction between agents is used to
ensure that risks are appropriately managed. The use of
Des Greer
des.greer@qub.ac.uk
a risk register is presented where a list of risks triggered
in the project is displayed at a dashboard. The big advan-
Darryl Stewart
dw.stewart@qub.ac.uk
tage of the approach is that software agents can be used to
detect risk and react dynamically to changes in agile pro-
1
Queen’s University Belfast, University Road, Belfast, UK ject environment. To validate the approach some innovative
2
Faculty of Computer and Mathematical Sciences, Universiti case studies using student projects are described. Evidence
Teknologi MARA, 77300, Merlimau, Melaka, Malaysia is therefore provided for the feasibility and applicability of
13
Vol.:(0123456789)

824 E. E. Odzaly et al.
the approach and finally some conclusions and discussion when it is done for the first time (Deemer et al. 2010) and
is given. there are issues with agile skills and personnel turnover, as
well as job dissatisfaction (Boehm and Turner 2003; Mel-
nik and Maurer 2006; Melo et at. 2011). In Scrum indi-
2 Research problems vidual motivation is very important and influences how
diligent team members are; for example in attending daily
2.1 Traditional risk management scrum meetings (Hossain et al. 2009). Recognising non-
compliance with established practices can provide early
Risk management in research papers is always acknowl- signs of risks e.g. low morale expressed during the daily
edged as being of utmost importance. To determine what meeting or avoiding discussing problems when behind
is needed we used existing work (Odzaly et al. 2009) on schedule (Lindvall et al. 2002).
an investigation of the barriers to risk management. The Due to the fact that agile methods depend a lot on the
results of that investigation concluded: credibility of the people involved in the projects (Cockburn
and Highsmith 2001; Nerur et al. 2005) as well as their
• That there is no standard or commonly adopted risk motivation in applying the agile practices (Layman et al.
management process and/or tool being used in software 2006; Conboy et al. 2010), most issues encountered relate
development situations. to the people and the practices involved. This echoes one
• That Risk Identification was the most effort intensive of the values in agile manifesto i.e. “individuals and inter-
process and additionally 30% agreed that risk monitor- actions over processes and tools” (Fowler and Highsmith
ing is most difficult and needs more effort. 2001). This implies that not having the right people doing
• That the biggest barrier was that visible (and tangible) the right process will be a source of risk.
development costs get more attention than intangibles Cho (2008) developed some research work on issues and
like loss of net profit and downstream liability. challenges of agile software development with Scrum. The
• Despite the acceptance that risk management methods research work mainly aims to provide guidelines to assist
enhance system development, nonetheless little sup- the companies to avoid and overcome barriers in adopting
port is to be found on the provision of these methods the method. An in-depth study was conducted between two
(Ropponen and Lyytinen 1997). It has been argued that different companies using various qualitative data collec-
the methods of managing risk in software development tion methods. The study presented various common catego-
are not comprehensive as they deal with specific types ries of issues and challenges in agile projects, among which
of risk (Bandyopadhyay et al. 1999). Besides, despite the following points are discussed in Table 1.
many well known risk management approaches having Chakradhar (2009) has outlined the common pitfalls
been introduced, risk management was still reported as in agile projects in which, he proposed that it is vital for
not being well practiced (Ibbs and Kwak 2000; Pfleeger the project manager to understand those pitfalls in order to
2000). As reported in Bannerman (2008) the most com- reduce risks. Among the important aspects discussed are:
mon risk management approaches found in the literature failure to provide sufficient training in agile methodologies;
highlight practices such as checklists, analytical frame- unfamiliarity of the project manager with agile methods;
works, process models and risk response strategies. poor involvement from the Product Owner; the team prac-
Many researchers have conducted research in tailoring ticing ‘single expert’ with no knowledge sharing as well as
risk management, providing various approaches. How- having passive team members.
ever, only a few studies have reported integration of risk Cockburn and Highsmith (2001) highlighted that one
management with contemporary software development of the most important success factors in a project is indi-
(Nyfjord and Kajko-Mattsson 2008). vidual competency. They emphasize the qualities of people
involved in the project, where good people will complete
the project while if team members do not have sufficient
2.2 Risk issues in Agile software projects skill, no process can compensate for their deficiency. This
is also supported by Boehm and Turner (2005) where peo-
It is clear that people issues are the most critical in agile ple issues are the most critical and it stated as very impor-
projects and that these must be addressed if agile is to be tant to address them before adopting and integrating agile
implemented successfully (Boehm and Turner 2005). practices into a project. That paper presents a list of barri-
Indeed, one of the most important success factors in an ers to using agile methods successfully and among the sig-
agile project is individual competency (Cockburn and nificant issues highlighted with respect to people are their
Highsmith 2001). Additionally, estimation of effort is a roles, responsibilities and skills as well as their ability to
consistent challenge in agile development work, especially predict and be knowledgeable.
13
Agile risk management using software agents 825
Table 1 Categories of issues/challenges in agile projects (Cho 2008)

Categories Issues found
Human resource Formation of team, where team was being organized without considering their necessary skills and knowl-
edge
Multiple responsibilities where one team member is responsible for many tasks
Some developers are not aware of the benefits of applying agile methods, and so are reluctant to apply
agile practices
Lack of accountability where team members do not take responsibility for any delayed task coupled with a
lack of supervision
Collaboration between team members is difficult, especially when they are not located together
Structured development process Scrum meetings; daily scrum, sprint planning and sprint review meetings are sometimes inefficient where
they are being held too often, or taking up too much time or setting up the meeting is difficult
Difficulties in estimating project work on legacy code
Environmental Poor customer involvement and unclear product requirements
Individual contribution is not recognized and no guidelines exist in determining accurate measurement of
individual performance
Information systems and technology A lack of communication between team members that are co-located causes duplication in resolving
problems
Newly hired team members tend to create code errors due to unfamiliarity with the software
Deemer and Benefield (2006), discuss common chal- aggressive datelines, having to work overtime as well as
lenges in Scrum. One of the challenges put forward is the underestimation of completion of time.
ability of a team to provide estimation of effort in their Nelson et al. (2008) introduced a risk management tech-
development work especially when it is done for the first nique that can be adopted in agile processes. The paper also
time. Most teams fail to deliver the tasks committed to due provides an argument in relation to agile being risk driven
to poor task analysis and estimation skills. When this hap- in that it implicitly manages risk in the process. One of the
pens, the team tends to extend the duration of the sprint implicit techniques used is to prioritize tasks at the start of
rather than learn to do the correct estimation. This can iteration. However, simply placing higher priority to a high
cause problems in achieving a sustainable pace since the risk task is not considered as managing risk. It does reduce
team will not be able to work reasonably due to delay in the risk to the project if the associated task is executed ear-
completing other tasks in the project. lier, but until the risk is resolved or no longer applicable,
Having a team member that is an agile sceptic, meaning, the task needs to be monitored for the risk, and action taken
they are opposed to agile methods, can have a huge impact if necessary. When identifying the right risk for the task
to the team as a whole. This is due to the fact that an agile and analysing it is not done properly, presenting an appro-
team relies heavily on trust and sharing of tacit knowledge priate plan to mitigate the risk is difficult (Williams et al.
to support important practices like pair programming and 1997; Ahmed et al. 2007).
shared ownership (Boehm and Turner 2005). Melo et al.
(2011) presents an unusual result with regard to the rela-
tionship of pair programming to tasks and motivation. Sur- 3 Solution approach
prisingly, whether the tasks are too easy or too complex,
may influence the motivation to work in pairs. As a result of the issues identified, there is a strong
Another important practice in agile process is the col- motivation to improve the management of risk in agile
lective code ownership. The study results discussed in Lay- projects without reducing agility in projects. In reality,
man et al. (2006) indicates that collective code ownership contemporary risk management should be able to be inte-
provides benefits in terms of knowledge sharing within the grated into the agile process to support decision making.
team. However, the disadvantage of this is that the team This includes taking into account human factors such as
will tend to choose the fastest solution ignoring its quality, developer skills and ability as well as their behavior in
assuming that they are not the only one who is responsible performing tasks. Due to the fact that Agile relies heav-
for the quality or otherwise of the code. By not assigning ily on the competency of the people involved, therefore
task ownership for a piece of work, this could demotivate we converted these issues to risk factors i.e. situations
the team in writing code that conforms to the standards or events that may cause a loss to occur and therefore
or required quality levels. Other problems discovered are that we need to monitor in a project as shown in Fig. 1.
working at a sustainable pace and accuracy of estima- In order to establish the basis of the ART model in rela-
tion. These problems are due to the situations like having tion to risk management, a model called Goal-driven
13

3.1.1 Input
The model represents how risks are gathered and managed

throughout an agile project. During the Input stage, the
agile process begins with planning and requirements gath-
ering. At this stage, while preparing the project, at the same
time, the gathering of risk data can commence. Require-
ments in agile processes are most often represented as user
stories. These are textual descriptions that contain the cus-
tomer’s specification of needs for the required system. A
product backlog is a subset of these requirements that will
be selected from based on priority.
The environment data used contains:
Fig. 1 Agile risk tool (ART) GSRM Model
• A project in this context is a set of user stories, the
membership of which is not fixed at any point of its life-
time. Each project relates the unique project name of the
project, a set of goals for the project, when it started and
when it ended.
• A team is a set of persons where each person consists
of a set of attributes describing the person. Each team
is working to achieve the goals of the project. For each
team member there is specific information, for example
on the type of skills that the team member possesses
and also their levels of expertise in defined skills, stated
as an integer;
• User stories are divided into tasks. A task refers to a tex-
tual description of the task associated with the estimated
Fig. 2 Agile risk tool (ART) model describing the application of risk hours of completion, the name of the person responsible
management in Agile environment
for the task and the progress for the task;
• Progress refers to additional information on the progress
software development risk management model (GSRM) of a specific task as reported by the person responsible
proposed by Islam (2009) was reused. This model is for the task. This includes information on attendance
used to specify the input for the project, which consists of the team member in the daily scrum meeting and
of the type of risks and the risk indicators as well as the whether progress or an impediment is reported for the
environment data which can be used to identify the risks task;
for the project. Thus the issues discussed earlier are • Risk data represents information on risk captured by the
transformed into a set of sprint goals. These will later be tool. The information includes the name of the risk, its
used to define the risks and their indicators thus allow- severity, the owner of the risk, location of the risk as
ing risks to be monitored continuously. well as the date the risk is triggered and resolved.
The following section will discuss the development of
the ART model and how it is used in dynamically man- The risk indicators and rules refer to a set of predefined
aging risk. risk factors brainstormed by the team at the early stage of
the project and encoded as rules (this will be further dis-
cussed in the next subsection). The risk indicators contain
3.1 The agile risk tool (ART) model a textual description indicating a threshold or state that will
trigger the risk. One example might be where a high prior-
The development of the ART model started with the ity task is selected in the sprint by a developer with too low
establishment of a view of how risk management may a predefined skill threshold. Rules contain a list of condi-
apply in an agile environment. Figure 2 below depicts an tions for an event encoded into IF/THEN statements. Later,
overview of the resulting model. this information is stored in the rule engine. Input data
refers to a set of collected data from the environment and
translated into a set of templates readable by the tool.
13
During the Process stage, the project proceeds as itera-

tions which include sprint backlogs, design and code, test-
ing and small releases of the product requirement. Itera-
tions contain are time-boxed into fixed length durations of
development. Risk agents (or ART agents) will manage the
risk based on the input data defined earlier. This risk pro-
cess is autonomous, where software agents; identify, assess
and monitor risk based on the input data from the environ-
ment. Once any risk is triggered, risk data will be displayed
in the Risk register. Any changes or updates to the envi-
ronment will affect the risk data (whether or not the risk is
flagged up). Fig. 3 Risk decomposition graph for the Agile risk tool (ART)
At the Output stage, the final risk data can be obtained agents of four risk management activities
after the delivery of the product and during a Sprint review
meeting. The risk register provides a view of all identified decomposition of risk into conceptual elements like risk
risk data. At the end, the data displayed in the Risk Register factor, risk event, risk outcome, risk reaction, risk effect
can be recorded and saved in the risk data repository where and utility loss. More recently a top down goal decomposi-
this information can be used to plan future projects. tion technique is described in Bresciani et al. (2002) and
The model has been demonstrated further and used as Dardenne et al. (1993). Indeed Boehm’s tutorial on risk
part of the work in (Odzaly et al. 2014). This is where the (Boehm 1989) decomposes risk management into activi-
ART architecture proposed was demonstrated in order to ties. In this work the category or type of agents used was
explore the application of risk management in agile appli- derived based on initial agent goal decomposition as shown
cations. This paper complements that in focusing on the in Fig. 3, based on Boehm’s work.
development of ART agents used at the Process Stage. The generic aim of this work is to find ways of lowering
the barriers to application of risk management. One of the
• The development of ART Agents objectives is to use the agents since agent behaviour is more
adaptable and can act on behalf of the project manager of
One way to move towards automation is to give software the agile project. In this case, some of the effort of the pro-
agents responsibility to identify, assess and monitor risk. ject manager is replaced by agent execution such that they
These agents ideally should be able to autonomously react will react automatically according to their own goals. In
to environmental changes, where the environment in this identifying goals for the agents, the top level goal is started
case is the software development environment, including in order to apply risk management in software development
the set of tools being used. project, particularly in agile projects. This goal is further
In order to reduce barriers in risk management applica- decomposed into two intermediate sub goals; assessing risk
tion, a lightweight risk management approach is needed. and controlling risk. These sub goals are then decomposed
The newly proposed approach includes three main steps into six smaller sub goals; identify, analyse, prioritize, plan,
in risk management; risk identification, risk assessment resolve and monitor. As a result of the decomposition of the
and risk monitoring. The rationale of doing so was twofold goal, agents were assigned based on the smallest sub goals
(1) to develop a realistic and acceptable risk management which supported the top level goal. Since the most effort
process that can fit into the agile methods (2) an empiri- intensive steps identified earlier were identification and
cal study (Odzaly et al. 2009) confirmed the most compli- monitoring, for the meantime, both sub goals were selected
cated steps in managing risks were risk identification and in addition to analyse and prioritize goals as highlighted
risk monitoring. In addition, prior to this section evidence in Fig. 3. Note that here that only the bottom level goals
is established that contended that risk management was dif- are engaged; the assumption being that top and intermedi-
ficult mainly due to the required human effort. Given this, ate level goals might have largely a controlling function but
the aim is to substitute some of the human involvement nonetheless have their own goals on how lower level agents
with autonomous software agents with the goal that these should interact.
could manage risk and minimize the need for manual input. Further ART agents were developed for this work as four
Automated agents can therefore help ease the work load in agents; manager agent, identify agent, assess agent (com-
managing risk, specifically in identifying, assessing and bines analyse and prioritize goals) and monitor agent. This
monitoring risk. is depicted as in Fig. 4 that shows the interactions (com-
Decomposition of risks into activities is common- munications via passing message) between manager agent
place. One example discussed in (Kontio 1997) used and the identify, assess and monitor agents. Depending on
13

Fig. 5 Sniffer Agent
Fig. 4 The communication between the ART agents and how they

interact within the environment data and rule engine triggered as informed by the ART template. The ART
agents will initiate communication between them. Mes-
sages are passed according to request and each agent will
the data from the environment, the agents react to detect notify another agent in prompting any further action to
risk dynamically through rules execution, where rules are be taken. An example of the ART agents’ communication
invoked from the rule engine. The ART agents’ communi- was introduced earlier in this chapter (Fig. 4).
cation is described further as below. Figure 5 show an interaction between the ART agents
There are four ART agents and each of them has a desig- starting when a risk is triggered. The figure shows the
nated goal assigned to them. The goal and purpose of each agents passing message using the Sniffer agent in the
of these is discussed below. JADE platform. True to its name, sniffer agent is a purely
java application that tracks messages in the JADE envi-
• Manager Agent acts as an intermediary between the ronment. It is useful when debugging the agent behav-
other three agents. It manages and executes rules, gets iours and for analysing message passing using in the
data from the Environment and notifies Identify agent if sniffer GUI (Bellifemine et al. 2007).
any risk is triggered. Rules and the environment data are dynamically edit-
• Identify agent is notified if any risk is triggered. It able. In the event where changes need to be made, one
requests from the Manager agent what risk has been can modify the environment data (which has been trans-
identified and notifies the Assess agent. lated into the ART template earlier) as well as the risk
• Assess agent is invoked by the Identify agent and its rules and indicators using the provided main screen area.
goal is to estimate the risk exposure (RE) of the identi- On the other hand, when developing possible risks asso-
fied risk where RE = probability (P) × impact (I). The ciated with rules and risk indicators, one might find the
identified risk will then be ranked as high, medium or environment data used to be insufficient to detect certain
low and the monitor agent is notified to take subsequent risks. In some cases, a small change in collection of the
action. environment data would allow defining or detecting more
• Monitor agent is invoked by the assess agent with some risks. For example, adding the information on developer’s
data: RE and rank of the identified risk. The monitor skill will allow monitoring the developer’s programming
agent will establish the location of the identified risk capability especially in completing high priority task.
along with the owner of the risk. This data is then dis- An example of a rule syntax that can be used is, “IF the
played in the Risk Register which later can be recorded developer skill level is ‘low’ AND the developer involved
and saved in the risk data repository. with a ‘high’ priority task, THEN there is probability a
risk of the task cannot be completed on time because of
the developer’s poor programming skill”.
3.1.2 Process ART agents will react dynamically to input data, pro-
cess the input by assessing any risk triggered and produce
At the Process Stage, the ART agents will monitor the a risk result in the Risk Register.
risk by acknowledging any rules or risk indicators
13
Functions
env_data_onChange(): This function exists for each

env_data item and is triggered when the sensed value
has changed
checkRiskCondition(env_data) is a function that com-
pares the value of env_data with the appropriate indica-
tor and returns a set of applicable rules from the ruleset,
risk_rules[ ]
applyRule() calls any necessary actions (e.g. emails
alerts)
Fig. 6 Agile risk tool—Risk Register updateRegister() appends or updates risk in the risk_
register[ ]
3.1.3 Output updateDashboard() notifies the dashboard of new/mod-
ified/deleted risks.
The idea of a Risk register has been defined by Wil-
lams (1994) who states that “the risk register has two
main roles. The first is that of a repository of a corpus of Algorithm for each env_data:
knowledge… The second role of the risk register is to ini- env_data_onChange()
tiate the analysis and the plans that flow from it”. While BEGIN
(Patterson and Neailey 2002) reported that very few applicableRules :=
development and construction of risk registers although checkRiskCondition(env_data)
ForEach rule in applicableRules
it is commonly used in Risk Management. As such, risk
BEGIN
register developed in this work can represent as a risk applyRule()
dashboard in which one can see a list of risks triggered updateRegister()
by the ART agents. The Fig. 6 shows an example of risk updateDashboard()
register used as the visualization of output in this tool. END
An overview of the main algorithm is described below: END
Input
env_data = Environment_data items (project, team,

task, progress, risk) as sensed from tools used. Here
we will refer to a generic type env_data. 4 Case studies
risk_indicator[ ] = sets of threshold values for each
type of environment data. There is a 1:N association In order to show that the suggested approach is feasible
between each env_data type and risk_indicator. Each case studies were conducted using student project artefacts.
risk indicator represents a measurement than can be
sensed in the environment for that environment data 4.1 Case study methodology
item.
risk_rules[ ] = User defined ruleset defining for each This work aims to answer the following Research Ques-
risk_indicator how to react when the value of env_ tions (RQs):
data meets/exceeds/falls below risk_indicator. There
is a 1:N association between risk_indicator and risk_ 1. RQ1: Is it possible to support risk management in agile
rule[i]. projects using existing SE environment data to over-
come barriers in the application of risk management?
Output 2. RQ2: Can data collection be conducted with minimal
intrusion and effort?
risk_register[ ] = set of risks where risk is a textual 3. RQ3: Can software agents coupled with a rule engine
statement and a conceptual value (high, medium, low) provide a means to automate risk management in agile
Dashboard = display of risks projects using data from the software development
environment?
13

In order to address these, case studies were used using likely to be costly. Menzies et al. (2009) use the term ‘data
the proposed solution approach and tool support. The drought’ to describe the situation where there is an unwill-
choice of case study was based on the fact that the devel- ingness from organizations to share their operational data
opment of the approach was rather new and exploratory due to, among other factors, business sensitivity associated
involving a large number of variables; thus a flexible and with the data. Given the difficulties of obtaining industrial
natural method was needed. Easterbrook et al. (2008) data coupled with the ready availability of student project
declared exploratory case studies to be suitable for pre- data in a university setting, student group projects were
liminary investigations of some phenomena to develop new used in order to demonstrate and validate the approach. On
hypotheses and build theories, and confirmatory case stud- the other hand, the use of student project data has strength-
ies to assess the existing theories. Case studies can provide ened the approach due to the access to a unique data set
a deeper understanding on the subjects under study and the which is not be available in any other setting. This includes
results obtained are valuable as well as contributing to the the access to data for a set of parallel agile teams all carry-
body of knowledge (Kitchenham et al. 1995). ing out identical sets of user stories. This scenario would
Easterbrook et al. (2008) promotes case study research be very difficult to find or engineer in a similar study else-
where it uses the purposive sampling, thus depending on where, and virtually impossible in an industrial setting.
the nature of the research objectives. A case study is suita-
ble when research is required in order to gain deep insights 4.2 Case study design
into chains of cause and effect. Furthermore, exploratory
investigations are appropriate where there is little control The study used data from a final year undergraduate hon-
over variables in the study. It started with the investigation ours course in Agile Methods, taught at Queens University
into issues and problems in risk management in agile pro- Belfast in the years 2011 and 2012. In the theoretical part
jects in order to gain deeper understanding of the phenom- of the course, students received lectures on general agile
ena followed by a proposed solution approach. Later, the development practices with an emphasis on Scrum. Dur-
solution approach is validated using a developed prototype ing the course, students were required to build a large soft-
tool using the case studies labelled CSA and CSB. The first ware artefact using Microsoft.NET technologies using an
case study, Case Study Alpha (CSA) was considered neces- industrial strength environment adopting both agile project
sary to explore the problem domain and was preliminary and software engineering practices. This includes applying
in nature, mainly intended to develop the Research Ques- important Scrum project management practices such as Pair
tions above. The second case study, Case Study Beta (CSB) Programming, Test Driven Development, Release and Itera-
was conducted as a confirmatory case study and was used tion Planning and Refactoring in their software project. The
to assess the existing theories and results developed from first case study was developed in 2011 involving 38 under-
CSA. However, both case studies aimed to provide vali- graduate students, assigned into six groups with six or seven
dation of the solution approach and the tool support with developers each. All groups were required to develop the
improvements being made in CSB, based on the lessons same product requirements. The first case study was devel-
learned from CSA. Mixed methods were used includ- oped in 2011 (CSA) with groups labelled Alpha1 to Alpha6
ing (1) an informal interview with the Product Owner to (Alp1–6). The second case study was developed in 2012
validate results from the prototype tool and (2) artefact or (CSB) and involved a total of 56 undergraduate students
archive analysis was done to understand compliance with with eight groups, Beta1 to Beta8 (Bet1–8) each group con-
agile practices in the team as well as to demonstrate the sisting of five to eight developers. All groups were given the
outcomes generated from the prototype tool. The artefact same product requirements as in the first case study. How-
or archive analysis refers to the investigation of project data ever, due to some missing data from group Bet8 this group
which includes the Agile Project Management tool spread- was dropped from the study, leaving only 48 undergraduate
sheet they used to collect data, minutes of the meeting and students involved in this study. The projects in both case
SVN repositories in order to identify patterns in the behav- studies involved two sprints, SP1 and SP2 which respec-
iour of the development team. tively had at least 10 to 15 working days. Before the start
The case studies were carried out on groups of students of the project, the Product Owner, a role played by a mem-
who were tasked with a software project and used their ber of academic staff, introduced the Product backlog items
data as an input to the proposed approach. Many practi- which consist of a list of prioritized user stories. Thereaf-
tioners in agile projects claim that agile methods inherently ter, it was each group’s responsibility to deliver these to the
reduce risk (Boehm and Turner 2005; Cohn 2005) but to satisfaction of the Product Owner. The Product Owner and
the authors’ knowledge very little research has been done to students were able to communicate regarding any arising
confirm this or in relating these two areas. However intro- issues in both sprints; therefore the groups have been super-
ducing something new in an organization is difficult and vised throughout the development process.
13
Table 2 The summary of No. Environment data Attributes Value Used Repository

collected environment data from
the student projects in Case 1 Project ID Project unique number ✓ Project data
Study Alpha (CSA) and Case
2 Project status Not started, in progress, completed ✓ Project data
Study Beta (CSB)
3 Team name/ID Team member name or unique number ✓ Team data
4 Role ScrumMaster /developer ✓ Team data
5 Total No. of role assigned 1 or 2 roles ✓ Team data
6 Total No. of project involved 1 or more projects ✗ Team data
7 Team skills Programming (C#) ✗ Team data
8 Agile experience True/false ✓ Team data
9 Agile level Very good, good, average, poor, very poor ✓ Team data
10 Skill level 5 (highest skill) to 1 (lowest skill) ✓ Team data
11 Task Name/ID Task name or ID ✓ Task data
12 Task priority High, Medium, Low ✓ Task data
13 Paired by Paired or Not Paired [“ ”] ✓ Task data
14 Total owned 1, 2 or more developers ✓ Task data
15 Estimated hours No. of hours ✓ Task data
16 Daily meeting attended Yes/no ✓ Progress data
17 Progress details Yes/no ✓ Progress data
As described in Sect. 3.1, the case studies were used to template was set up for collecting data from these catego-
demonstrate the ART Process Flow (Fig. 2) which contains ries. The collected data from the student projects was as far
essential stages in Input, Process and Output stage. The as possible, screened and matched in order to meet the gen-
most vital part of the process is to determine its environ- eral cases from the studied tools. Even though the student
ment data and risk rules for the project. This is further elab- projects did not use any of the studied tools the same envi-
orated on in the following sections. ronment data was available in their environment by other
means. The summary of the possible environment data that
4.2.1 The environment data can be collected is simplified in the following Table 2. Note
that there are two available data items that were not used
At the beginning of this work, two agile project management in this study; Total number of projects involved and also
tools were studied; Extreme manager1 and Rally software2 in Team skills since the students in the case studies were only
order to define possible environment data that may be availa- involved with one project at a time and all students fulfilled
ble in a real-world scenario and could be used in this work. the required skill in programming, which in this case was a
The environment data were categorized as follows: need for C# experience. It is envisaged that in an industrial
project this data would also be used to identify risks.
• Project data—contains information about the project i.e.
project name, start date and end date. 4.2.2 The risk rules
• Team data—contains information about the team mem-
bers i.e. skills and experience. In previous studies, the research problems and issues in
• Task data—contains information on the user stories and agile projects were discussed and transformed into a set
breakdown of the tasks associated with the estimated of problem scenarios which was then presented in Fig. 1.
hours. Each problem scenario represents a possible risk event that
• Progress data—contains information about the team is associated with a Sprint goal for the project. The Sprint
reports on task progress. goal is important since it can be used to consider how envi-
ronment data values could be used as indicators of threats
After this information has been obtained from the stud- to those goals i.e. triggers for the risks. Therefore, it is pro-
ied tools, four categories of data described above were posed that risk rules can be formulated using the risk indi-
focused. Upon starting up the first case study, the ART cators to identify events that cause loss (delay/extra cost/
loss of value) i.e. risks, leading to a situation where risk
identification can be automated.
1
Hindsa, Extreme Manager, http://www.hindsa.com, accessed 14 Table 3 below show the sets of risk rules and risk indi-
March 2014. cators for the problem of task ownership. This problem is
2
Rally, http://www.rallydev.com, accessed 14 March 2014.
13

Table 3 Rule template for task ownership

Goal G1: In sprint X, task Y should be assigned to appropriate number of developers once Sprint X is started
Problem scenario PB1: during the sprints, the developer does not have any pair or has too many programming partners for the selected task
Consequences Avoiding ‘single expert’ or too many developers sharing code
Indicators IN1.1: Project is started and when a task is selected in the sprint, ‘task paired by’ is empty—indicates high risk
IN1.2: Project is started and the selected task owned by more than 2 developers—indicate low risk
Repository/data Project data
Task data
Rule(s) RL1.1: If Project.Project_Status = [In Progress]
AND If Task.Paired_By = [“”]
RL1.2: If Project.Project_Status = [In Progress]
AND If Task.Total_Owned > 2
Risk ID and Name RN1.1: R0001 pair programming
RN1.2: R0002 task ownership
The remaining project goals; G2: Skills and Experience, G3: Resources and G4: Progress, are not discussed further in this paper. However, the
generated risk rules associated with goals described earlier in Fig. 2 were mapped to goals as follows (Table 4). In each case the risk represents a
threat to the goal success
Table 4 Mapping goals to associate Risk ID used in Case Study the particular risks. As mentioned earlier, the set of indica-
Alpha (CSA) and Case Study Beta (CSB) tors and rules can be updated from time to time depending
Goals Risk ID how the project manager decides to identify risk. One such
case is the modification of the rule for pair programming
G1: task ownership R0001, R0002 which took place between CSA and CSB.
G2: skills and experience R0003, R0004,
R0009,
R0019
G3: resources R0005, R0006 4.3 The case studies
G4: progress R0007, R0008
Given that the approach is novel and still at a research
stage, the chosen study was rather exploratory in nature,
inferred from the risk issues related to people and using where it relies on the collection of existing data used in the
the Rule template presented in Table 1. Earlier, the ART project as opposed to ongoing data collection in an ideal
GSRM model was developed that shows the relationship situation. Where possible, multiple sources of evidence
between goal, risk-obstacle and assessment. Using this (triangulation) were used, meaning that archival artefacts
model, the following rule template table (Table 3) show and informal interviews with the product owner were used
one of the sets of goals, problem scenarios, rules and risk to confirm findings. For example, after each collection of
indicators used in this work. data an informal interview with the product owner was held
Table 3 indicates the set of rules and risk indicators in order to verify the interpretation based on the collected
for goal G1 where when a sprint is started, an appropriate data. As mentioned earlier, students had no knowledge that
number of developers should be assigned to the particular their project data was being be assessed for risk, removing
tasks. The usage of indicators generally depends on how any possible bias in this respect. Rather they were moti-
the project manager wants to signify that the condition vated in demonstrating that they had followed agile pro-
appears to be at risk. Two indicators were selected for the ject management practices e.g. pair programming as taught
goal based on the risk issues highlighted earlier. Indicator in classes and producing high quality working software.
IN1.1 states that once the Project Status was [In Progress] Based on the data collected in CSA, some issues were
and the Task selected has no pair [“ ”], i.e. null or empty found in adopting the approach. These were noted along
string, then the Risk 0001—“Pair programming” is trig- with conclusions of further discussions with the product
gered. IN1.2 states that once the Project Status is [In Pro- owner. The outcome from this investigation was recorded
gress] and the Task is owned by more than two developers in the investigation notes. Further, one modification was
[>2] then the Risk 0002, “Task ownership”, is triggered. made to one rule, R0001, to be used in the next case study.
These indicators are then translated into rules RL1.1 and The following section discusses in detail the ART pro-
RL1.2 that contain object, attributes and value of the attrib- cess flow (Fig. 2) as conducted in the two case studies.
utes that will be picked up by the agents. The repository Both case studies consisted of two sprints so that the pro-
section shows where the environment data is involved for cess flow was repeated iteratively in four instances.
13
Table 5 The environment No. File name Data available/collected

data extracted from the student
project artifacts 1 Hartmann-Orona Spreadsheet Sprint backlog information
Major task area/user stories
Task name
Task owner
Task status (completed/in progress/not started)
Estimated hours
Commits days and hours for each task
Team member information
Team member name and initials
Working days for this sprint
Working hours for this sprint
Start date
End date
No. of calendar days
Sprint team member daily activity
Team burndown chart
Team member burndown chart
2 Sprint backlog target List of user stories
Points for each user stories
Dependencies
3 Scrum minutes of meeting (Daily) Name of the scrum master for the day
Work progress for each team member
Work done since yesterday
Work planned today
Problems
4 TortoiseSVN repositories Directory and files versioning
Commit files/code
Track changes
5 Source code (C#) integrates with resharper Group project source code
Code quality analysis
4.3.1 Case study alpha (CSA) • Sprint Backlog Target—contains a list of user stories
and associated points and dependencies.
As described earlier, the ART Process Flow consists of • Scrum Minutes of Meeting—contains information on
three main stages; input, process and output. In this section team attendance and updates on tasks.
this process is described in detail. • Code Repositories—a subversion (http://subver-
The Input stage starts by gathering all necessary data sion.tigris.org) source control system that was used
from the student project artefacts and translating the data by students to manage their project. Each group was
to match the ART Template. During the Input stage, there required to log their activities and check in all docu-
were two inputs needed—definition of the list of environments and source code.
ment data and definition of the risk rules used in this study. • Source Code—the students were required to use the
In order to define the list of environment data, two steps C# and VB.Net programming languages
were performed.
Table 5 below shows the list of extracted data from the
1. Gathering data archived artefacts of the environment data.
The archived artefacts available however, did not pro-
For the purpose of defining the list of environment data, vide as much data as the studied commercial tools. Nev-
archived artefacts derived from the student projects were ertheless, the archived artefacts contained enough useful
used. There were five main artefacts used in this study, information, particularly related to sprint backlog and
summarised as below. the user stories, breakdown of the tasks, details of the
developer responsible for a task and so on. In addition,
• Hartmann-Orona Spreadsheet—a well known spread- the goal of the study was to demonstrate the approach
sheet mostly containing sprint backlog data, including and tool support, not applicability to every data item col-
the breakdown of each user story into tasks, estimated lected in mainstream tools.
hours and owner of each task.
13

the ART template. This includes transforming the raw data

Environment Data ART Template obtained from the artefacts in the form of object oriented
HartmannOrona.xls Object.Properties [Attribute]
[TaskName, TeamName] Team.TeamName[Sarah,John] concepts.
Minutes.doc Task.Progress [Yes,No]
SprintBacklog.doc … This is essential so that the ART agents will be able
to pick up the data and match this with the rules embed-
ded in the tool. Since this study was done after the project
had ended, the data obtained was comprehensive start-
Fig. 7 Translating environment data to ART template
ing from day one in sprint SP1 until day 15 in sprint SP2
and ready to be translated into the template. In the event
One issue that had to be overcome was that of missing where the project is new or ‘fresh’, data can be keyed or
data in the SVN repository. Although all student groups added directly through the user interface. Similarly, there
were required to log their work into the repository, some were also issues highlighted while doing this step. The
groups had not done this. For example, all groups were main issue identified was associated with the process of
required to record daily minutes of meetings yet some of translating the raw data into the template. Since this was
the minutes were missing from the repositories. Conse- done manually, it involved tedious and highly effort inten-
quently, the product owner had to trace this record through sive tasks. In this case study, there were six student groups
other methods such as email archives to obtain the miss- with six different sets of archived data. Even though they
ing data. Similarly, there were some inconsistencies in might have the same format or almost the same format of
the format of the minutes of meetings. For example each the document, the interpretation and explanation of the data
group should have specified the name of the Scrum Mas- was different. To overcome this it was often necessary to
ter for each meeting. However, this information was miss- carefully go through all the documents in the repositories
ing in some of the groups. Again, the product owner had in order to understand how they implemented their project.
to retrieve through email archives and provide this infor- Additionally, the problem is compounded in the case of
mation. All issues found were recorded and written in the tracking a task assigned to a team member. For example,
investigation notes so that the process could be improved in in the Hartmann-Orona spreadsheet, the team defined the
the future. user stories and the breakdown of the tasks. The spread-
sheet itself did not provide a unique id for each of the tasks
2. ART data translation although it did include the name of the person responsible
for the task. Whilst in the team minutes for each meet-
Once all data had been defined and organized into its ing each team member provided updates on the task they
categories, the next step was to translate this data manu- were assigned, this was only briefly described. In the event
ally to fit the ART template. Figure 7 shows an example where which team member committed to a specific task
of the translation of the data from the archived artefacts to had to be identified and were tallied with the spreadsheet,
Table 6 List of risk name along with its associated rules and probability and impact score
Risk ID Risk name Rules [Object.Attribute] = [Value] Prob score Imp score
R0001 Pair programming PROJECT.PROJECT_STATUS = Completed 3 5

TASK.PAIRED_BY = “”
R0002 Task ownership PROJECT.PROJECT_ 1 1
STATUS = Completed
TASK.TOTAL_OWNED > 2
R0003 High priority task assigned to inappropriate team member TASK.PRIORITY = High 5 4
cannot be completed on time TEAM.SKILLLEVEL = 1
R0005 Overload tasks can cause difficulty in time management PROJECT.PROJECT_STATUS = Completed 5 1
TEAM.TOTAL_NO_ROLE > 1
R0007 Developer absent in meeting possible of employee turnover PROJECT.PROJECT_STATUS = Completed 1 3
PROGRESS.DAILY_MEETING = N
R0008 No progress report PROJECT.PROJECT_STATUS = Completed 1 3
PROGRESS.PROGRESS_DETAILS = N
R0009 Unable to understand agile process and meet the target PROJECT.PROJECT_STATUS = Completed 3 3
TEAM.AGILE = false
R0019 Unable to comply with the agile process TEAM.AGILE = true 1 1
TEAM.EXPERIENCE = Very Poor
13
there is a need to go through the repositories and look at After each sprint in CSA was completed, reports were
the code commits by the team member. The time consumed created. The presented reports provide useful information
was 2–3 h for translation of one group’s data, excluding on the total of risk score each day and in one sprint. This
time spent retrieving missing data. includes information on the breakdown of risk identified
Next, the risk rules for this case study were defined. each day.
Table 6 below shows the list of risk name and rules as dis-
cussed at the beginning of this chapter as well as its prob- 4.3.2 Case study beta (CSB)
ability and impact score as defined for each risk. Note that
the rules were embedded in the Rule engine during the Based on the experience from CSA, investigation notes
development of the ART prototype tool and at this stage revealed the following issues:
existing rules from the Rule engine database had only to
be selected. However, when needed, new rules were added 1. Design of the project: Since the project was designed
into the Rule engine or existing rules edited using the pro- for students as part of a university course, the real goal
vided user interface. Similarly, when entering the probabil- in practice was for students to apply what they had
ity and impact the parameters could be adjusted later on. learnt during the course. Hence changing the structure
For this case, the value of the probability and impact score of the project was not possible. Further, due to the lim-
for each risk was cross checked with the Product Owner. itation of time in completing their project it is not pos-
Since this was a student project, there was no actual impact sible to add more management tasks. Nonetheless, data
on cost involved for this project. Therefore, the impact fac- collection needed to be easier.
tor was based on the consequences of the student not com- 2. Format of the document: In order to avoid missing
pleting the project and not producing a quality end product data, standard formats were established for documents
as required by the Product Owner. In the real world project, used for data collection in the project; i.e. meeting
the risk identified will be more project-specific, in other minutes.
words risks are assessed individually for a specific project 3. Naming conventions/traceability: It was decided that
situation or environment. A significant project risk can be in order to easily track the data between a task and its
the result of certain characteristics in the project environ- owner a unique id for each task and each team member
ment. For example, a developer who is considered to have was required. For example, all tasks should start with a
very low skills but is assigned to a high priority task could unique id beginning with “TS” e.g. TS001 and all team
lead to a higher risk exposure compared to where that members could have a unique id starting with “TM”
developer is assigned to a lower priority task. In brief, the e.g. TM001.
project manager determines what risk is significant and 4. Task allocation and estimation: Based on the summary
how severe the risk is. of data collected on each group project background,
The Process stage is the stage where the risk assess- it is found that some groups failed to allocate tasks
ment automatically took place. Based on the defined evenly to each team member. Some team members car-
inputs described previously, the ART agents communi- ried out too many tasks which resulted in some of the
cate between the ART template and the Rule engine. At tasks not being completed. Further, it was noticeable
this stage, once the project is loaded into the ART proto- that some of the estimated task sizes were too big and
type, changes can be made using the provided user inter- should have been split into smaller tasks of almost the
face. Once the tool is ‘run’, the ART agents will react if same size. It was also emphasized that team members
any of the rules are triggered and then notify an identified should practice pair programming whenever a bigger
risk. Any changes in the inputs will result in changes in the task is assigned.
outcome of the triggered risk as well. This is because the 5. Specifying student skill level and agile experience:
identified risk can be observed in the Output stage and the During sprint SP1 it was proposed that student skill
problem of ignoring a risk is avoided. and agile experience should be taken into account. At
In the Output stage, once a risk is identified, the risk this stage, identifying student skill was relatively easy.
result is displayed in the Risk Register. This should also Specifying agile experience is more problematic. In
show an overview of all risks triggered. This includes the SP1 it is assumed all students did not possess any agile
risk name, location of the risk associated with the affected experience but their agile experience was then meas-
task and the owner of the risk, defaulted to the owner of ured in SP2 based on the assessment by the educator
the task. The risk register also displayed the risk result from the first sprint.
according to priority, starting from the risk with high to 6. Risk rule for pair programming: Risk results were pre-
low severity. After one sprint is completed, the risk result is sented to the product owner and one of the most com-
stored into the Risk data repository. mon risks found was related to absence of pair pro-
13

Table 7 Risk name along Risk ID Risk name Rules [Object.Attribute] = [Value] Prob score Imp score
with its associated rules and
probability and impact score R0001 Pair programming PROJECT.PROJECT_STATUS = Completed 3 5
(modified) TASK.PAIRED_BY = “”
TASK.ESTIMATE > 5
gramming. It was obvious, from the findings that most addition, it is agreed that to implement this approach the
of the students did not adhere to this practice. How- first time was rather challenging. This was greatly improved
ever, the argument was that some of the tasks were too and the process would be more effortlessly managed if
small and were not suitable for work in pairs. As such, adopted repeatedly.
it is essential to propose to modify the rule, where the Next, the risk rules for this case study were applied. In
modification being described further in the following the previous case study, the list of the risks and their asso-
case study—CSB. ciated rules (Table 6) were summarized. Based on the out-
comes and lessons learnt from the previous case study, the
Based on the lessons learnt, the improvements to data pair programming rule was modified in order to provide
collection were made and to performance of the students a more realistic risk rule. The ability to modify the rules
in applying the agile practices learnt in their course. Stu- as needed demonstrate that the solution approach and tool
dents were informed of these. At this stage, it is assumed support is dynamically responsive to changes thus, as is
that the product owner’s awareness of the students’ perfor- required in agile projects. This is described in Table 7
mance has increased based on the lessons learnt from CSA. which shows the highlighted risk rule as modified. The
In addition, it is expected to change the performance of the remaining risk rules were unchanged.
student group in this case study. Once the Process and Output stage have been completed
During the Input stage, the same steps included in the for the two sprints, again, the reports on the information
previous case study were adhered. As described previously, gathered on the Risk data were created and presented in the
due to the nature of the project we were not able to change form of diagrams. The outcome of both case studies has
the structure of the project and therefore the same artefacts yielded results of Total Risk Score (TRS).
were used and data capture methods from these artefacts.
At the data gathering step the process was found to be
4.4 Case study results
much easier than in the previous case study. For example,
some of the student groups used the new naming conven-
Risk data derived using the tool and displayed in a Risk
tion in naming their reports and one group used the format
Register was recorded and saved in the Risk Data Reposi-
of the proposed minutes of meeting. As reported earlier
tory. This risk results later were assessed and presented
regarding some missing data from the previous student pro-
using graphs.
jects and for this case study there was one group who had
not logged their main document which was the Hartmann-
Orona spreadsheet in the repositories. This document was 4.4.1 Summary of CSA project data
untraceable therefore there is a need to drop this group
from this study. Although the acceptance in changing the During the planning phase, each team was given Product
working method is rather poor in this instance this can be backlog items that consisted of a list of user stories. They
improved gradually. Similarly, in a real world project it is were asked to estimate the work effort needed for each
normal that some organizations might refuse or feel chal- user story in hours and break this down into a set of tasks.
lenged when asked to change their methods. However, Hence, each team had a varying range of total number of
due to the limitation discussed earlier it is not possible to tasks in each sprint. At this stage, the students were chal-
compel the students to adhere to standards due to the con- lenged, based on lectures taught in class, in their ability to
straints of this also being an assessment exercise. plan and estimate the work effort required for each sprint.
As discussed earlier the difficulty in carrying out the The output from the tool is used as means of assessing
process of transforming the raw data into the template the total risk in the project at any point or in a post sprint
during the ART data translation step. Although the effort review as shown in Figs. 4 and 5. This graph includes infor-
of performing this step is still quite intensive, the time mation on the breakdown of risk identified each day using
to translate one group’s data was reduced to less than an total risk score (TRS). TRS is based on the generic severity
hour. This was especially the case where the standard nam- score of a risk item for the task it is related to. The metric
ing convention was used in reporting for some groups. In provides results on counting of risk daily and cumulative
13
CSA - TOTAL RISK SCORE IN SP1 CSA - TOTAL RISK SCORE IN SP2
60 110
100
50 90
80
40 70
30 60
Tot al Risk Score
Tot al Risk Score

50
20 40
30
10 20
10
0 0
1 2 3 4 5 6 7 8 9 10 Day 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15Day
Alp1 2 13 25 36 38 31 42 34 53 46 Alp1 0 2 21 26 36 50 59 62 60 69 97 76 73 72 103
Alp2 2 14 13 33 32 35 39 41 17 7 Alp2 16 5 4 3 15 18 18 15 17 16 12 10 8 9 7
Alp3 3 0 0 23 27 18 19 21 37 48 Alp3 6 7 10 17 13 13 23 24 40 26 13 9 14 12 10
Alp4 10 14 13 24 31 29 30 13 18 12 Alp4 3 7 5 10 10 12 12 13 15 12 11 13 18 8 7
Alp5 10 23 14 20 15 15 22 41 23 32 Alp5 0 0 2 3 6 10 2 2 3 7 1 3 4 2 3
Alp6 0 6 10 13 13 16 31 26 13 23 Alp6 5 4 3 4 5 4 7 5 6 14 9 11 5 2 6
Fig. 8 Total risk score graph of case study alpha (Sprint 1) Fig. 9 Total risk score graph of case study alpha (Sprint 2)
risk counting in a sprint for continuous risk management BREAKDOWN OF TRS ALP2 IN
purposes. The TRS is calculated as below. SP1 & SP2
Consider that there is a set of tasks T in the project on a
SP1 SP2
given day d: 45
Td = {t1…tn} where n = total number of tasks on day d. 40
35
There is also a set of predefined generic risks R that can
Tot al Risk Score pe rday
R0009/19
30
potentially be identified in the project: 25
R0008
R = {r1…rm} where m = total number predefined risks. 20

R0007
R0005
Thus, risks associated with each task, t on a given day d 15
R0003
is Rtd ⊂ R. 10 R0002
The TRS for a task t on day d is therefore. 5 R0001

0
TRStd = card (Rtd), where card is the cardinality of the 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25
set. Day
These risks can be associated with any of the tasks t in

Fig. 10 Graph showing the breakdown of Total Risk Score for team
T. The risks are present while the task is being carried out
Alp2 in SP1 and SP2
in the sprint.
Therefore, for a given day d the TRS is
graphs show the increasing and decreasing pattern of risk
t=n
∑ detected for each team with some peaks at certain days
TRSd = TRStd ,
which give a better visualization of risk.
t=1
The already-established risk burndown chart (Cohn
where the total of risks triggered will be calculated in all 2010) generally results in a downward risk exposure graph,
tasks carried out in that particular day. computed from the changing probability and size of poten-
The application of TRS can be used in a current or on- tial losses in every sprint. However, that technique does
going project or a past project. In a current project, TRS not show or visualize specifically which risk is being iden-
is calculated daily. This is when the project manager can tified, assessed and monitored throughout the project. On
see the risk triggers from day one and if it is resolved, the the other hand, the graph below may be used to develop a
risk is no longer appears. On the other hand, TRS can be risk trend for a type or a category for agile team. For exam-
applied to a past project as a means for review. The risk ple, a team who are new to agile projects, one might see
data obtained can be used as an input using the proposed risks might increase gradually to a peak and start decreas-
approach or to predict risk for a future project. ing towards the end of the sprint as risks are resolved or
Figures 8 and 9 below show a plot of the TRS for both tasks completed. On the other hand, a more established
sprints. In SP1, the number of risks score ranged from team might not have any risk occurring daily in their sprint;
as low as zero (either no risk being present or no ongo- instead showing a peak at certain times with regards to the
ing task) up to the highest of 53 risks found in team Alp1, type of risk that they want to monitor.
Day 9 of the sprint. In SP2, risk score ranged from zero to The data shown in Fig. 10 provides a better visualization
the highest of 103, also found in team Alp1, Day 15. Both of the type of risk triggered each day in team Alp2, for both
13

CSB - TOTAL RISK SCORE IN SP1 CSB - TOTAL RISK SCORE IN SP2
45 40
40 35
35 30
30 25
Tot al Risk Scor e

Tot al Risk Score
25
20
20
15 15
10 10
5 5
0 0
1 2 3 4 5 6 7 8 9 10 Day 1 2 3 4 5 6 7 8 9 10 Day
Bet1 5 6 13 3 4 13 9 14 6 4 Bet1 1 7 8 5 9 4 9 8 4 0
Bet2 1 11 22 29 32 22 18 18 20 13 Bet2 0 0 0 0 2 1 3 5 8 8
Bet3 4 9 16 20 19 23 32 21 30 31 Bet3 14 7 3 4 3 7 4 4 3 2
Bet4 0 6 11 12 15 26 19 17 17 1 Bet4 0 8 2 1 2 5 4 4 6 3
Bet5 8 7 8 15 12 15 13 19 18 8 Bet5 1 2 2 0 4 1 2 2 3 1
Bet6 4 14 12 9 9 19 10 11 11 2 Bet6 1 2 2 3 3 4 6 5 4 4
Bet7 9 22 29 23 19 41 28 43 28 24 Bet7 1 2 9 14 11 18 18 33 38 15
Fig. 11 Total risk score graph of case study beta (Sprint 1) Fig. 12 Total risk score graph of case study beta (Sprint 2)
sprints SP1 and SP2. The advantage of this method was

twofold; (1) one would be able to view the category of risk
triggered for the particular day as well as (2) one would be
able to view which risk being triggered the most and needs
attention from the project manager in order to help in mak-
ing decisions e.g. whether prompt action should be taken
towards the risk that occurred consecutively. For example,
in SP1, team Alp2 had generated risk mainly through not
performing pair programming and from having team mem-
bers without agile experience. However, in SP2, since the
students had already developed using agile in SP1, the risk
of no agile experience no longer existed, but the risk from
not performing pair programming could still occur. In the
student projects these risks if realized might have very little
impact towards project success but in the real-world pro- Fig. 13 Graph showing the breakdown of total risk score for team
Bet5 in SP1 and SP2
ject and these risks are triggered without being addressed
a threat to the project could arise. On the other hand, pat-
terns of risk occurrence for a particular team member can Figures 11 and 12 show the calculated TRS for both
be inferred and used for estimating future project risks in sprints. In SP1, the risk score ranged from as low as zero
similar projects with the same team members. (either no risk being present or no ongoing tasks) up to
the highest risk score of 43 in team Bet7, Day 8 of the
4.4.2 Summary of CSB project data sprint. Whereas in SP2, the risk score ranged from zero to
the highest risk score of 38, which was also found in team
Comparing with the TRS presented in the CSA earlier Bet7, Day 9 of the sprint. Both graphs show the increasing
(Figs. 8, 9) for both sprints, the TRS for CSB was much and decreasing pattern of risk detected for each team with
lower than the TRS in CSA. This has supported the some peaks on certain days thus providing us a better visu-
assumption stated earlier that the indication of possible alization of the risks arising. Normally one would expect
inspection with regard to risk may affect the developer or risk should decrease (burn down) over a sprint. However,
student behaviour during the project. Despite the modi- referring to both figures of a sprint review, for Team Bet1
fication of the pair programming rule for CSB, the TRS on both sprints would demonstrate clearly a useful out-
produced in this case study was more realistic so that pair come of using the tool, that the team performance here was
programming was critical only in bigger tasks. This may be problematic.
due to the fact student projects were used in the case study Similarly to CSA, in the light of viewing specific risks
and a less motivation to do pair programming than if told that triggered on a particular day, the data shown in Fig. 13
to in a workplace. Thus, a high number of risks have arisen provides better visualization of the type of risk triggered
due to the violation of this rule. In practice, the project each day for team Bet5, for both sprints. The advantage of
manager could modify this rule from time to time. having this method is that we would be able to view the
13
type of risk triggered for the particular day as well as to in project environment data. This is discussed throughout
view which risk is being triggered the most and therefore Sect. 3.1.1 starting from defining the input, processing the
which should be countered as a priority. input and producing the output. Later, a walkthrough of the
ART process is adopted in both case studies supported by
the prototype tool. This demonstrates that software agents
5 Discussion coupled with a rule engine can automate risk management
using data from the project environment.
The results obtained from the case study have offered evi-
dence that apart from the novelty of the proposed approach 5.1 Study validity
and tool support, the approach is usable and provides
useful data. The TRS graphs produced in both case stud- Since the study presented introduces a new approach in
ies provide a useful visualization method which supports managing risk in agile project, the main issues are focused
identification and monitoring of risks in a dynamic agile on the internal threats. The first internal threat is in terms
project. They show the number of risks picked up by the of the accuracy of the measured data, especially because
ART agents and provide a realistic and interactive way of the data used was based on historical artefacts. Further,
monitoring risks. The study revealed that it is possible to confirmation of this data was not possible as the project
use existing SE environment data to support risk manage- had already been completed at the time of analysis. Sec-
ment. However, there are many environment data items that ondly, the approach used entailed manual collection and
can be used to detect risk thus, acquiring project manager translation of data from archived artefacts into the ART
to define which ones that are related to their project. Fur- tool. This human effort was required before the ART agents
ther to this, our empirical evidence also revealed that data could begin reacting towards environment data as they
can be collected with minimal intrusion and effort. were designed to work in. This effort could be minimized
Three Research Questions (RQs) were established ear- by selecting a proper individual in the team to conduct this
lier as an expression of the aims of this research work. process, for example the Scrum Master in a Scrum project.
The following paragraphs summarize how these have been One step taken to ensure the quality of the study was
responded to in the case studies. that cross checking was done from time to time with the
RQ1: The conclusion provides evidence that it is possi- Product owner to confirm perception based on his obser-
ble to use the project environment data to identify risk and vation. Considering external validity threats, the risk man-
so overcome the main barriers in the application of risk agement approach and tool supports were designed to be
management. This supports the notion that, in agile pro- as general as possible so that this is applicable in general
cesses, a lightweight risk management approach is required to agile project environments. This includes taking into
that automates some of the risk processes. As a result of account two popular agile project management tools stud-
this, a solution was developed using software agents to ied for this work so that the approach is as applicable as
react to the project environment, based on designated rules possible to other contexts but also lightweight and unobtru-
in order to manage risk. sive to the team daily activities. Nonetheless, no claim can
RQ2: The method used for both case studies evidenc- be made of good fit with tools not studied. Additionally, the
ing that data collection conducted from both case studies study used student project data along with the case study
involved minimal intrusion and effort, and with no cost execution guideline (Runeson and Höst 2009) rather than
involved. In the cases studies employed, the environment industrial data. Hence, there will be arguments whether this
data used includes the student project data, in this instance is applicable to a real world environment.
from archived data. The data was stored in SVN reposito-
ries and was retrieved by the educator for use in the case
study. Both case studies did not include any interaction 6 Conclusion
with the participants in the study setting in order to meet
the purpose of the study. This includes to identify risk in In this paper we presented a novel approach to manage risk
their project and to investigate compliance of the team in agile projects. The work offers contributions in two areas
member with agile practices. The collected data was vali- (1) on the use of case studies for assessing new methods
dated by the educator/product owner of the student’s pro- and tools and provides an example of how student teams
ject based on discussion sessions prior to and after the can be used to gather information not feasible in industrial
implementation of the project. settings. (2) on the use of agents to semi-autonomic ally
RQ3: A prototype tool was developed in order to vali- manage software risk.
date the interaction between agents, agents’ compliance This work provides several significant investigations on
with the designated rules and how agents react to changes the problems and issues in risk management specifically in
13

agile projects. The development of the ART model and tool Bellifemine FL, Caire G, Greenwood D (2007) Developing multi-
support has been demonstrated to help by at least reducing agent systems with JADE, Wiley.com
Boehm BW (1989) Tutorial: software risk management. IEEE
the problems previously identified with risk management. Computer Society Press, Washington
The approach is necessarily supported by a prototype tool Boehm B, Turner R (2003) Using risk to balance agile and plan-
which has been shown to manage risks in example agile driven methods. Computer 36(6):57–66
projects. The role of risk management in iterative and agile Boehm B, Turner R (2005) Management challenges to implement-
ing agile processes in traditional development organizations.
processes has to date been neglected but this model inte- Softw IEEE 22(5):30–39
grates risk management model with agile methods in a way Bresciani P, Perini A, Giorgini P, Giunchiglia F, Mylopoulos J
that does not bloat the agile process. (2002) Modeling early requirements in Tropos: a transforma-
This approach however, to the authors’ knowledge and tion based approach. In: Agent-Oriented Software Engineering
II. Springer, Berlin, pp 151–168
understanding has never been applied in risk management, Chakradhar K (2009) Risk management in agile development.
especially with the specific aim of reduction of human White paper edn. Polaris Software Lab Limited
effort. In addition, the resulting risk management process Cho J (2008) Issues and Challenges of agile software development
is naturally lightweight since each software agent is design with SCRUM. Issues Inf Syst 9(2):188–195
Cockburn A, Highsmith J (2001) Agile software development, the
to achieve a designated goal i.e. to identify, assess, prior- people factor. Computer 34(11):131–133
itize or monitor risk. This paper has led to use designated Cohn M (2005) Agile estimating and planning. Pearson Education,
software agents to facilitate the risk management process. London
Therefore, this work demonstrates the potential of autono- Cohn M (2010) Managing Risk on Agile Projects with the Risk
Burndown Chart
mous computing being applied to risk management where Conboy K, Coyle S, Wang X, Pikkarainen M (2010) People over
software agents have been used to assist the human oriented process: key people challenges in agile development. IEEE
and complex risk management process. In future, this work Software
aimed to comprehend the physical implementation of the Dardenne A, Van Lamsweerde A, Fickas S (1993) Goal-directed
requirements acquisition. Sci Comput Programm (20)1:3–50
ART model and tool support, where there is a need to inte- Deemer P, Benefield G, Larman C, Vodde B (2010) The scrum
grate this with existing Agile Project Management tools, primer. Scrum primer is an in-depth introduction to the theory
perhaps as a plug-in, so that automated risk management and practice of Scrum, albeit primarily from a software devel-
can be fully realised. This would allow more practical risk opment perspective, vol 1285931497. http://assets.scrumtrain-
inginstitute.com/downloads/1/scrumprimer121.pdf
management while a project runs in the foreground, soft- Easterbrook S, Singer J, Storey M, Damian D (2008) Select-
ware agents are in the background ready to manage emerg- ing empirical methods for software engineering research. In:
ing risks. Guide to advanced empirical software engineering, Springer,
Berlin, pp 285–311
Acknowledgements This research has been funded partly by Uni- Fowler M, Highsmith J (2001) The agile manifesto. Softw Dev
versiti Teknologi MARA Malaysia and has been part of the Ph.D the- 9(8):28–35
sis work submitted to Queen’s University Belfast, United Kingdom. Hindsa, Extreme Manager, http://www.hindsa.com. Accessed 14
This paper has been selected among best papers presented at the First March 2014
EAI International Conference on Computer Science and Engineering Hossain E, Babar MA, Paik H, Verner J (2009) Risk identification
(COMPSE) 2016. and mitigation processes for using Scrum in global software
development: A conceptual framework. Software Engineering
Conference, 2009. APSEC’09, Asia-Pacific, IEEE, p 457
Open Access This article is distributed under the terms of the Ibbs CW, Kwak YH (2000) Assessing project management matu-
Creative Commons Attribution 4.0 International License (http:// rity. Project Management Journal 31(1):32–43
creativecommons.org/licenses/by/4.0/), which permits unrestricted Islam S (2009) Software development risk management model: a
use, distribution, and reproduction in any medium, provided you give goal driven approach. In: Proceedings of the doctoral sympo-
appropriate credit to the original author(s) and the source, provide a sium for ESEC/FSE on Doctoral symposium, ACM, p 5
link to the Creative Commons license, and indicate if changes were Kitchenham B, Pickard L, Pfleeger SL (1995) Case studies for
made. method and tool evaluation. Softw IEEE 12(4):52–62
Kontio J (1997) The RISKIT method for software risk management,
version 1.00. Computer Science Technical Reports, University
of Maryland, College Park, MD, USA
Layman L, Williams L, Cunningham L (2006) Motivations
References and measurements in an agile case study. J Syst Archit
52(11):654–667
Ahmed A, Kayis B, Amornsawadwatana S (2007) A review of tech- Lindvall M, Basili V, Boehm B, Costa P, Dangle K, Shull F, Tesoriero
niques for risk management in projects. Benchmarking: Int J R, Williams L, Zelkowitz M (2002) Empirical findings in agile
14(1):22–36 methods. In: Extreme Programming and Agile Methods—XP/
Bandyopadhyay K, Mykytyn PP, Mykytyn K (1999) A framework for Agile Springer, pp 197–207
integrated risk management in information technology. Manage- Melnik G, Maurer F (2006) Comparative analysis of job satisfaction
ment Decision 37(5):437–445 in agile and non-agile software development teams. In: Extreme
Bannerman PL (2008) Risk and risk management in software pro- Programming and Agile Processes in Software Engineering
jects: a reassessment. J Syst Softw 81(12):2118–2133 Springer, pp 32–42
13
Melo C, Cruzes DS, Kon F, Conradi R (2011) Agile team perceptions Odzaly EE, Greer D, Stewart D (2014) Lightweight Risk Manage-
of productivity factors. Agile Conference (AGILE), 2011 IEEE, ment in Agile Projects. In: SEKE (pp 576–581)
pp 57 Patterson FD, Neailey K (2002) A risk register database system
Menzies T, Williams S, Elrawas O, Baker D, Boehm B, Hihn J, to aid the management of project risk. Int J Project Manage
LumK, Madachy R (2009) Accurate estimates without local 20(5):365–374
data? Softw Process Improv Pract 14(4):213–225 Pfleeger SL (2000) Risky business: what we have yet to learn about
Nelson CR, Taran G, de Lascurain Hinojosa L (2008) Explicit risk risk management. J Syst Softw 53(3):265–273
management In: International Conference on Agile Processes Rally, http://www.rallydev.com. Accessed 14 Mar 2014
and Extreme Programming in Software Engineering. Springer, Ropponen J, Lyytinen K (1997) Can software risk management
pp 190–201 improve system development: an exploratory study. Eur J Inf
Nerur S, Mahapatra R, Mangalaraj G (2005) Challenges of migrating Syst 6(1):41–50
to agile methodologies. Commun ACM 48(5):72–78 Runeson P, Höst M (2009) Guidelines for conducting and reporting
Nyfjord J, Kajko-Mattsson M (2008) Integrating risk management case study research in software engineering. Empir Softw Eng
with software development: State of practice. In: Proceedings, 14(2):131–164
IAENG International Conference on Software Engineering. Willams TM (1994) Using a risk register to integrate risk manage-
BrownWalker Press, Boca Raton Citeseer ment in project definition. Int J Project Manage12(1):17–22
Odzaly EE, Greer D, Sage P (2009) Software risk management bar- Williams RC, Walker JA, Dorofee AJ (1997) Putting risk manage-
riers: An empirical study. Empirical Software Engineering and ment into practice. IEEE Soft 14(3):75–82
Measurement. ESEM 2009 pp 418–421
13

Odzaly2018 Article AgileRiskManagementUsingSoftwa PDF

Uploaded by

Copyright:

Available Formats

Odzaly2018 Article AgileRiskManagementUsingSoftwa PDF

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Odzaly2018 Article AgileRiskManagementUsingSoftwa PDF

Uploaded by

Copyright:

Available Formats

J Ambient Intell Human Comput (2018) 9:823–841

Agile risk management using software agents

Abstract Risk management is an important process in 1 Introduction

Table 1 Categories of issues/challenges in agile projects (Cho 2008)

The model represents how risks are gathered and managed

During the Process stage, the project proceeds as itera-

Fig. 4 The communication between the ART agents and how they

env_data_onChange(): This function exists for each

env_data = Environment_data items (project, team,

Table 2 The summary of No. Environment data Attributes Value Used Repository

Table 3 Rule template for task ownership

Table 5 The environment No. File name Data available/collected

the ART template. This includes transforming the raw data

R0001 Pair programming PROJECT.PROJECT_STATUS = Completed 3 5

Tot al Risk Score

Td = {t1…tn} where n = total number of tasks on day d. 40

R = {r1…rm} where m = total number predefined risks. 20

The TRS for a task t on day d is therefore. 5 R0001

These risks can be associated with any of the tasks t in

Tot al Risk Scor e

sprints SP1 and SP2. The advantage of this method was

You might also like

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

Table 1 Categories of issues/challenges in agile projects (Cho 2008)

Fig. 4 The communication between the ART agents and how they

Table 2 The summary of No. Environment data Attributes Value Used Repository

Table 3 Rule template for task ownership

Table 5 The environment No. File name Data available/collected