AACE Journal - Cost Engineering July - Aug 2023

T H E J O U R N A L O F A A C E I N T E R N AT I O N A L | T H E A U T H O R I T Y F O R T O TA L C O S T M A N A G E M E N T
J ULY / AUG U ST 2023
Implementing
Qualitative
Project Risk
Management
ALSO:
The Importance of Owner/Contractor

Transparency Across All Contract Types
Implementation of a
Project Data Warehouse
ABOUT US
Established in 1956, AACE International
CORE is the Association for the Advancement

of Cost Engineering. Together with the
VALUES AACE International Certification Institute,
it serves over 10,000 members and
QUALITY Demonstrated by our
commitment to excellence and certificants in 100 countries.
continuous improvement
INTEGRITY Demonstrated by WHY WE EXIST

transparent and ethical policies AACE empowers its members to perform with confidence. Together,
and practices we make an impact and influence outcomes.
PROFESSIONALISM Demonstrated
by our leadership and knowledge WHAT WE DO
TRUSTWORTHY Demonstrated We set industry trends. AACE advocates for its Body of Knowledge
by our dedication to accuracy and the people who employ it through iteration and innovation of
and reliability trusted technical guidance and meaningful collaboration.
WHO WE ARE
AACE is a community of professionals from various practice areas
including project controls, cost & commercial management, change
management, planning & scheduling, estimating, decision & risk
Learn more, management, asset management, and dispute resolution. Members

include owners, contractors, and consultants from many industries,
or join today! such as construction, manufacturing, engineering, infrastructure,

natural resources, power generation & utilities, and government.
EXPLORE OUR WEBSITE

CONTENTS
J ULY/ AUGUST 2023
TECHNICAL ARTICLES
9 Implementing Qualitative
Project Risk Management
JESSICA M. COLBERT, PSP AND KIMBERLY FORBES, PSP
17 The Importance of Owner/

Contractor Transparency Across
9 All Contract Types
STEPHEN L. CABANO
24 Implementation of a
ERIC VYSKOCIL
ALSO IN THIS ISSUE
3 AACE International Board of Directors
17 3 Cost Engineering Journal Information

4 Letter from the Editor
24 For additional industry news and updates,

you can always visit us at web.aacei.org.
J ULY/AUG U S T 2 02 3 1
The Top 10 Reasons
To Join AACE International
Ready to advance your career and begin enjoying the advantages
that our members enjoy? Whether you are an experienced cost
engineer or a student, we have a membership ready for you.
1 Time 6 Technical Development

Gain access to a wealth of resources that will save you time Increase your knowledge and expertise by joining one
and money! You’ll stay informed about the complexities of the of AACE International’s many technical subcommittees,
cost and management profession - plus you’ll have access to subcommittees, and Special Interest Groups (SIG’s) at no
discounts on educational programs, publications, and more! additional cost to members. Discuss industry problems
with your peers or help experts develop new and improved
techniques and practices for the profession.
2 Information
Locate thousands of technical papers and publications in the
Virtual Library. AACE’s database is keyword searchable for 7 Networking
quickly locating appropriate reference articles. By attending a local section or our annual Conference & Expo
for interesting speakers, informational tours, social dinners and
much more. The online Membership Directory is an excellent
3 Career source for a list of contact information on thousands of members.
Join one of our many technical subcommittees and participate
Members can post resumes at no additional cost in our Career
in the AACE Communities - a great way to tap into the collective
Center and keep your career on track through information
wisdom and experience of our worldwide membership.
sources such as our annual Salary and Demographic Survey of
Project and Cost Professionals.
8 Excellence
4 Learning Our certification programs are independently accredited by
the Council of Engineering & Scientific Specialty Boards. AACE
We offer numerous online learning courses on estimating and
certifications are a recognized credible standard in the cost
project management. The Approved Educational Provider
management field. A recent study shows that individuals with
program helps maintain high quality development courses and
an AACE Certification earn 17.4% more than their counterpart
providers. AACE also holds many seminars throughout the year.
without a certificate.
5 Resources 9 Discounts
Starting with the TCM Framework and Recommended
On products and services ranging from AACE International
Practices that are available for free only to members to our
Conference & Expo registration fees, archived webinars and
bi-monthly publication Cost Engineering featuring articles
presentations, certification examination registrations, and more!
for cost professionals around the world. Through the AACE
International website, the Cost Engineering journal is a great
current resource for members and as a member, you gain
access to an archive of past issues.
10 You!
We are your professional partner bringing you information
and support you can trust. Join and become part of a unique
network of individuals who are dedicated to improving the cost
JOIN TODAY! web.aacei.org and management profession.
2 J U LY/AU G U ST 2023
COSTENGINEERING
AACE INTERNATIONAL
BOARD OF DIRECTORS
PRESIDENT
Patrick M. Kelly, PE PSP
president@aacei.org ESTABLISHED 1958 | Vol. 65, No. 4 July/August 2023
PRESIDENT-ELECT
Scott A. Galbraith, PE CFCC MANAGING EDITOR Marvin Gelhausen
preselect@aacei.org mgelhausen@aacei.org
PAST PRESIDENT GRAPHIC DESIGN Little Fish Design Company

Shoshanna Fraizinger, CCP info@littlefishdesigncompany.com
pastpres@aacei.org
ADVERTISING SALES Business Development Coordinator
SECRETARY Joanna Boggs
Mark C. Sanders, PE CCP CFCC PSP +1.304.296.8444 x1122
secretary@aacei.org jboggs@aacei.org
TREASURER
Cindy L. Hands, CCP
+1.304.296.8444
treasurer@aacei.org
web.aacei.org
DIRECTOR TECHNICAL
Dr. Stephen P. Warhoe, PE CCP CFCC FAACE Hon. Life
TechDirector@aacei.org
DIRECTOR EDUCATION
Nakisa Alborz AACE® International - The Authority for Total Cost Management®
EdDirector@aacei.org
OUR VISION - To be the recognized technical authority in cost and schedule
DIRECTOR CERTIFICATION management for programs, projects, products, assets, and services.
Chris Caddell, PE CCP DRMP
OUR MISSION - The members of AACE® enable organizations around the
CertDirector@aacei.org
world to achieve their investment expectations by managing and controlling
DIRECTORS-AT-LARGE projects, programs, and portfolios; we create value by advancing technical
Husain Al-Omani, CCP CEP DRMP EVP PSP knowledge and professional development.
DirectorAtLarge1@aacei.org
Cost Engineering (ISSN: 0274-9696/23) is published digitally on a bi-monthly production schedule by AACE International,
Michael J. Bennink, PE CCP PSP Inc, 726 Park Avenue #180, Fairmont, WV 26554 USA. Copyright © 2023 by AACE International, Inc., All rights reserved.
This publication or any part thereof may not be reproduced in any form without written permission from the publisher.
DirectorAtLarge2@aacei.org Access to the bi-monthly Cost Engineering journal digital files is a benefit of AACE International membership and
requires a member login and password. There is no subscription service for the Cost Engineering journal other than AACE
Hannah E. Schumacher, PSP FAACE membership. Digital access is on an individual use basis and not available on any group access basis. Cost Engineering is
a refereed journal. All technical articles are subject to a review by the AACE International Cost Engineering Journal Review
DirectorAtLarge3@aacei.org Committee. Abstracts are only accepted in our annual AACE “Call for Papers” for our Conference & Expo. Accepted abstracts
must be followed up with a full approved manuscript that is presented and attendee evaluated at one of our Conference
EXECUTIVE DIRECTOR/CEO & Expo events. Top rated manuscripts will be considered for publication in the Cost Engineering journal. Any unsolicited
abstracts received at other times throughout a year will receive e-mail notice to submit in our next “Call for Papers.”
Debra L. Lally, CAE Copying without written permission of AACE is prohibited. E-mail requests for photocopy permission to editor@aacei.org.
dlally@aacei.org ADVERTISING COPY: Contact AACE International, Inc, 726 Park Avenue #180, Fairmont, WV 26554. Telephone: 304.296.8444,
extension 1122. E-mail: marketing@aacei.com for rates. Advertisers and advertising agencies assume liability for all
content (including text, representation, and illustrations) of advertisements published and also assume responsibility
for any claims arising and made against the publisher. The publisher reserves the right to reject any advertising that is
not considered in keeping with the publication’s mission and standards. The publisher reserves the right to place the
words “advertisement” with copy which, in the publisher’s opinion, resembles editorial matter. All advertising accepted for
publication in Cost Engineering is limited to subjects that directly relate to the cost management profession. Current rate
card available on request. COST ENGINEERING DEADLINES: Submissions for Cost Engineering must be received at least 30
days in advance of the issue date. Send to: Business Development Coordinator, AACE International, Inc, 726 Park Avenue #180,
Fairmont, WV 26554 USA. Deadlines do not apply to technical papers.
Policy Concerning Published Columns, Features, and Articles

Viewpoints expressed in columns, features, and articles published in Cost Engineering journal are solely those of the
authors and do not represent an official position of AACE International. AACE International is not endorsing or sponsoring
LEARN MORE AT: the author’s work. All content is presented solely for informational purposes. Columns, features, and articles not
web.aacei.org/about-aace/structure designated as Technical Articles are not subject to the peer-review process.
LETTER FROM THE
EDITOR
BY MANAGING EDITOR MARVIN GELHAUSEN
Building a
Technical Article
Summer has arrived in the US It is AACE International policy that technical papers published in the
Cost Engineering journal must first be submitted, accepted, and presented
and it is time for our annual at an AACE Conference & Expo. All accepted papers are published in
the annual Transactions. Only the top scoring papers are considered for
“Call for Papers” where publication in the Cost Engineering journal.
This month in the CE Journal, we are featuring the following three
authors can submit abstracts technical articles:
for presentation at the 2024 TECHNICAL ARTICLE 1

Conference and Expo. July and "Implementing Qualitative Project Risk Management,” is authored by Jessica
M. Colbert, PSP, and Kimberly Forbes, PSP. This risk themed paper was
August are the only months in first presented as RISK-3803.
which abstracts and/or technical TECHNICAL ARTICLE 2

"The Importance of Owner/Contractor Transparency Across All Contract
papers can be submitted. Types,” is authored by Stephen L. Cabano. This owner themed paper was
first presented as OWN-3820.
TECHNICAL ARTICLE 3
"Implementation of a Project Data Warehouse,” is authored by Eric
Vyskocil. This Data Science and Advanced Analytics (DSAA) themed
paper was first presented as DSAA-3798.
We asked each author to elaborate on how they selected their topic and
what they want you the reader to get as takeaways from reading their
articles. We also asked them about how they learned of AACE, why they
became members and what AACE member resources have best benefited
them personally and how AACE has impacted their careers. We hope this
additional background insights will give you a better understanding of “Risk management
each article.
does not have to
be complicated; risk
management can be
TA 1. Implementing Qualitative started at any point in
Project Risk Management the project.”
BY JESSICA M. COLBERT, PSP, AND KIMBERLY FORBES, PSP
KIMBERLY FORBES, PSP
MEMBER ENCOURAGEMENT
Both authors Jessica Colbert and Kimberly Forbes say they became an
AACE member because of “encouragement from existing members.” Jessica
was encouraged by Patrick M. Kelly and Kim by Chris Carson and Patrick
Kelly. These all worked together in 2010.
When asked what the most beneficial and useful resource AACE
membership offers her, Kim responds, “Hands down it is networking out into a wider variety of opportunities.” Kim believes her career has
and volunteer opportunities where you can increase your knowledge benefited by being an AACE member. She says, “I am part of MBP because
of AACE and form stronger connections to other members.” Jessica of the association due to meeting other employees at the conferences and
says, “Networking and exposure to technical content” have been most through volunteering for AACE.”
beneficial to her. One thing that Jessica believes will help AACE International is to,
“Encourage topic-specific networking opportunities where members
NETWORKING AND VOLUNTEERING can share what they’re currently doing to solve certain problem.” For
Jessica continues, “I’ve grown my skill sets in the areas of scheduling, example, she adds, “there was a great topic on the conference app about
claims, and risk management which has allowed my career to branch how people are leveraging PowerBI to support project controls functions.”
Kim adds that she would like to see AACE “Research into making
membership comparable financially for international members that
aren’t being paid by employers.”
“Qualitative risk Jessica and Kim have presented three AACE technical papers and
participated in one panel discussion. This is Kim’s first technical paper to
assessments are a great be published in the Cost Engineering journal. She says she selected the
topic based upon, “The need and current practice on projects.” Jessica
place to start if a project adds, “There was a gap in current papers available related to this topic.”
The content of the article is based upon overall practice on multiple
or organization is new projects. Jessica adds that, “All sources are referenced in the paper.” She
also says, “We used past experiences on multiple projects but not a
to risk management. specific project example.”
The takeaways Jessica wants you to get from her article are that: “Risk
Risk management is management should be implemented early but can be implemented at any
stage of the project. Qualitative risk assessments are a great place to start
an iterative process if a project or organization is new to risk management. Risk management
is an iterative process that must be actively managed and monitored.” Kim
that must be actively adds, “Risk management does not have to complicated; risk management
can be started at any point in the project; and the importance of building
managed and monitored.” an historical risk management practice.”
Kim says her article is primary targeted to the construction industry,
JESSICA M. COLBERT, PSP but it can benefit any reader to put into practice on a project in any
industry. Jessica adds that the article, “Can relate to any industry but the
article focuses on construction.”
CONFERENCE & EXPO EXPERIENCES
Jessica says she has presented at least 3 papers, plus at least 1 panel
participation. All of the articles have been published in Transactions.
At least one was published in Cost Engineering Journal. For the 2023
Conference & Expo, Jessica participated only in a panel discussion and
“AACE International
did not present another technical paper. Kim has attended at least seven
AACE International Conference & Expo events, including ones in Las
needs to be discussing
Vegas, San Diego; Orlando, Toronto, New Orleans, San Antonio; and
Chicago. For the recent 2023 Conference & Expo, she participated in a
the future trends in
panel discussion for CSC-4023 titled, “Project Controls - A Profession on the
Edge of Extinction or Rebirth?” She also presented paper CDR-4123 with
project control with the
Charlie Bolyard, due to the coauthor being unable to attend. This technical
presentation was titled, “The Schedule Delay Analysis for Subcontractors –
use of digital technology
Different by Necessity.”
(digital twins, BIM, etc.)
KEEP IT SIMPLE
Jessica says her advice to potential new authors is, “Keep it simple and ask
to attract the next
for subject matter experts to review it before submitting to AACE.” She
says presenting a paper is, “great exposure to share ideas and contribute to
generation of project
industry advancement.” Kim’s fully agrees with the “Keep it simple” advice
as the top tip for new author/presenters. She says a benefit of presenting
control resources.”
is that “it helps to get your name out in the organization and to peers. Also,
STEPHEN L. CABANO
attendees are encouraging and welcoming during presentations.”
YOU MAY NOT KNOW THAT . . .

Something you may not know about Jessica is that she is an “advocate of
continued education and professional development” and she “commits
time each year to teaching, mentoring, and coaching.”
Kim shares that currently in her private life, she is: “In the thick of having
two active children and my husband and I driving them to a practice AACE MEMBERSHIP HAS BEEN A REWARDING EXPERIENCE
almost every evening – wouldn’t change it for anything! Not rushing any Further commenting on the benefits of AACE membership, Stephen says,
chapter of their lives.” “The knowledge and expertise of the contacts I’ve made have helped me
become a better project manager and project controls professional and
have enhanced my abilities/expertise as a training specialist.” He adds, “I
have been honored to have been awarded an AACE Fellow and received
the Brian D. Dunfield Educational Service Award and the O.T. Zimmerman
Founders' Award. These achievements have enhanced my career.” He
TA 2. The Importance of Owner/ believes AACE International needs to be “discussing the future trends in
project control with the use of digital technology (digital twins, BIM, etc.)
Contractor Transparency to attract the next generation of project control resources.”
Noting that his employer, Pathfinder, collects “Hot Topics” information/
Across All Contract Types issues throughout the year from contacts and clients, projects we are
working on, and research we are conducting. We then compile this
BY STEPHEN L. CABANO list for timely topic ideas for our conference participation and training
sessions. This resource assisted Stephen in selecting his topic and
FATHER WAS AN AACE MEMBER preparing this article. In pulling together his article, Stephen says, “We use
For author Stephen Cabano association with AACE International has been research conducted by organizations such as the Construction Industry
a family experience. He says, “I was introduced to AACE International over Institute (CII), The Project Management Institute (PMI), the Engineering
30 years ago by my father who was also a member.” He further explains Construction Contacting (ECC) association and others. We have also
that he became a member, “because AACE is the premier association for used AACE’s Virtual Library, the TCM Framework®, Recommended Practices
project controls, cost & commercial management, change management, (RPs) and Cost Engineering journal.”
planning & scheduling, estimating, decision & risk management,
asset management, and dispute resolution practitioners. AACE’s body COLLABORATIVE RELATIONSHIPS BRING BENEFITS
of knowledge was developed and is used by industry professionals Stephen adds that, “Many past and current projects were used to compile
worldwide.” Stephen adds, “ I find the connections to other project control the information and data for this article.” He explains that “The paper will
professionals in our industry extremely beneficial as well as the access discuss the benefits of bringing contractors, suppliers, and the owners
to AACE’s Body of Knowledge (the TCM Framework®, Recommended closer together in a more collaborative relationship. Also, readers will
Practices (RPs) and Cost Engineering journal, etc.).” learn that project control transparency is critically important for any
selected contracting strategy. Finally, the paper will provide an outline
of a reasonable data request which, when correctly communicated, will
encourage a win-win situation for owners and contractors.” These are the
takeaways Stephen wants you the reader to get from this article.
He believes the focus of his article is targeted toward issues across
the heavy industrial sector of capital projects. He says, “Many of these
same issues are encountered in other industry sectors but this article is
focused on the oil refining, petrochemical, pharmaceutical, power, material “Pick a topic that the
processing, and other related process industry projects.”
industry is struggling
AVOID GETTING TOO ANALYTICAL
Stephen is an experienced author and presenter. He says, “I have with or that you can
presented 17 conference papers and this will be my fifth publication. I have
participated in the AACE International Conference for about 25 years.” He improve upon. Do
presented a paper titled “Project Control for Owners’ Small Project Portfolios”
at the 2023 conference. Willing to share advice and pointers to first time or not get too analytical
newer authors, he says, “Pick a topic that the industry is struggling with or
that you can improve upon. Do not get too analytical as the ultimate goal is as the ultimate goal is
to deliver successful projects, not to over analyze data.”
Something you may not know about Stephen is that he is “a ‘die hard’ to deliver successful
Philadelphia sports fan – including college basketball at my alma mater,
Villanova University.” He adds, “My wife and I enjoy all kinds of fishing projects, not to over
but especially deep-sea fishing for tuna and billfish.”
analyze data.”
STEPHEN L. CABANO
TA 3. Implementation of a
BY ERIC VYSKOCIL
NEW AUTHOR PRESENTS FIRST ARTICLE

Author Eric Vyskocil says he virtually attended the 2021 AACE
Conference and Expo and found the topics interesting and relevant. challenges but can implement different solutions. Without a forum like the
He says, “One of my managers, Tracy Leung, Director – Pickering AACE, we become like frogs at the bottom of that well, unaware of what
Redevelopment, encouraged me to become more involved in AACE and solutions have already been implemented successfully elsewhere.”
supported me when I wanted to present a technical paper about Ontario Eric says, “Organizations greatly benefit from being able to see real
Power Generation’s (OPG) experiences.” He adds, “I have presented world practical examples of challenges faced and lessons learned from
only a single conference paper to date, at the 2022 Conference & Expo. I projects.” He believes that “Finding ways to encourage members to share
found the experience very fulfilling and look forward to presenting more their experiences would help meet member needs.”
in the future.”
SHARING LESSONS LEARNED
AACE PROVIDES REAL WORL EXPERIENCES In explaining how he selected his topic for this article, Eric says, “OPG’s
Eric lists the real-world experience from members and organizations project historical database is something that I’ve been extensively involved
presented through technical papers, the Cost Engineering journal, with and when I saw that the AACE had recently published Recommended
the recommended practices, and conferences as being what attracted Practice 114R-20 on the same topic, I felt it was very timely for us to
him to AACE membership and what continues to be beneficial and share our lessons learned.” He further explains that he “did not use
useful resources. any non-AACE sites for researching content, as it primarily concerned
the lessons learned from our organization. I did speak extensively with
A FROG IN THE WELL individuals like Tracy Leung (Director – Pickering Redevelopment), Ron
In explaining how AACE membership has benefited his career, Eric says Hall (Senior Manager – Risk Management and Estimating) and Michael
“There is a story about a frog who lived his entire life at the bottom of a well Wang (Senior Manager – Project Reporting and Data Infrastructure) while
and only learns about the amazing world outside his well when told by developing the article to get their input.” The article is entirely based on the
a passing bird. Organizations that execute projects generally face similar experiences from OPG’s work to implement a project historical database.
SPEAK ABOUT YOUR EXPERIENCES
Eric’s advice to anyone who has not presented a paper but is considering
doing so is to: “Speak about your experiences.” He says, “Many of the
challenges we face executing projects are quite common across industries
and there often isn’t one correct answer. Other organizations are interested
“OPG’s project historical in hearing about real world application of processes and tools to solve
issues in ways that may be different than how they have.”
database is something Eric concludes: “My career has evolved in ways I didn’t expect when
I finished my graduate studies in physics, having moved from performing
that I’ve been physics analyses, to project management, report automation, database
development, application development and now managing a team of
extensively involved project controls specialists. Being willing to take on new challenges and
learn new skills has been essential and rewarding.”
with and when I saw
that the AACE had
recently published
Recommended Practice
114R-20 on the same
topic, I felt it was very
timely for us to share
our lessons learned.”
ERIC VYSKOCIL
BRIDGING THE GAP

In alerting readers what to look for in his article, Eric says, “We have seven
lessons learned in the article, but one key challenge that organizations
face with IT related work such as the development of a project historical
database is prioritizing the work that provides the most and longest
lasting value to the organization. People with the need often don’t know
what technical solutions exist, and people who are familiar with the
technical solutions don’t always know what will provide the most value.
Organizations need individuals who can bridge that gap and help drive
the right solutions at the right times.”
Eric believes any organization that executes projects could benefit
from this article. He says, “Data analytics requires organizations to
store historical project data so that trends can be investigated and
identified. Storing data from large numbers of projects requires some
kind of database.”
Eric did not present a paper at the 2023 Conference & Expo but says
he will be submitting one for the 2024 Conference & Expo. He adds, “At
OPG, we have performed a number of analyses investigating the predictive
power of different metrics, such as the IEAC, ETC and CPI, at different
stages in the project execution. I’m very excited to share the correlations
we’ve identified.”
Implementing
Qualitative
Project Risk
Management
BY JESSICA M. COLBERT, PSP AND KIMBERLY FORBES, PSP
ABSTRACT
Everyone can remember that one perfect, flawless project, right? No? Neither can the
authors. Projects are messy and challenging, full of risks that threaten the schedule
and budget. Risk management is a critical component to project success, yet so many
project teams and organizations fail to put it into practice. If an organization has no
formal risk program or is early in its implementation, project risk management can be
inconsistent and often ineffective. This article explores various strategies to implement
qualitative risk management on any project, discover how qualitative risk management
can be injected into typical project processes without consuming significant resources
or budget, and develop strategies for continued use throughout the project life cycle
and for historical management. These project-level implementation strategies can
later evolve to quantitative risk management at the project level and formal enterprise
level programs. Finally, the authors discuss common hurdles project teams experience
with risk management implementation and solutions to prevent or overcome these
challenges. This article was first presented as RISK-3803 at the 2022 AACE International
Conference & Expo.
risk and review previously addressed risk. The team will need to keep
Introduction the momentum going and require the stakeholders to think proactively.
Proactive thinking may be a different way of operating for some teams. The
Projects are messy and challenging, full of risks that threaten the schedule team may need to transition from simply reacting to events or issues as
and budget. Risk management is a critical component to project success, they occur to a state of proactively thinking about what the next thing may
yet so many project teams and organizations fail to put it into practice. If an be to create a negative impact or a positive opportunity to the project.
organization has no formal risk program or is early in its implementation,
project risk management can be inconsistent and often ineffective. RISK WORKSHOPS
Conducting dedicated risk workshops or, alternatively, risk interviews,
at each transition point of a project is another proactive project risk
management initiative. Is the project going from design development
Goals of Project Risk to construction documents? Hold a risk workshop. Is the project going
from sitework and foundations stage to superstructure stage? Hold a risk
Management workshop. Having gateways or milestones predefined is an easy way to
keep the team focused on continuous risk monitoring. By implementing
By engaging in some form of risk management, the team is demonstrating periodic risk workshops and reviewing top risks during progress meetings,
their commitment to protecting the project outcome. This is accomplished the team is set up for success.
by maximizing potentially positive impacts and minimizing potentially
harmful impacts to cost, time, safety, continuity of operations, or
quality. Risk management can improve the efficiency of project delivery
by stakeholders working together to mitigate risks, thereby providing Preparing for Risk Identification
satisfaction to the owner with a well-delivered project. Additionally,
sharing a common project goal of meeting established stakeholder and Analysis
objectives will aid in having a collaborative and high-performing team.
Engaging in risk management as a team can increase and improve COMMON UNDERSTANDING OF TERMINOLOGY
communication, enhance problem-solving, and develop innovations. The It is important for the project management team to have a common
team’s confidence in the accuracy of project forecasts will increase because understanding of basic terminology associated with risk management.
every team member will have contributed to forward looking thinking and Key terms to understand are as follows and can be found in AACE
planning. As the project progresses and performance is measured against International’s Recommended Practice 10S-90 [1] :
project objectives, the project risks continue to be identified and managed.
This process can reduce uncertainties and increase forecast accuracy. • Brainstorming
Overall project performance may also improve through these efforts if the • Concern
risk register is actively used to manage and mitigate risks. • Risk
• Opportunity
• Threat
• Uncertainty
Key Elements of Project Risk • Issue
• Impact
Management • Action Item
• Risk Event
BASIC PROCESS • Condition
Project teams may question when the best time to implement a risk • Risk Owner
process should be. Early and often is ideal, but no matter in what stage the • Risk Responses (Threats/Opportunities)
project is, risk management can begin. Risk management is a proactive, ◦ Avoid/Exploit
continuous, and iterative process. There may be few risks early on, but ◦ Reduce/Enhance
every risk identification cycle can present additional risks. A risk register ◦ Transfer/Share
is the primary qualitative tool that will be used to capture potential ◦ Accept/Accept
risk and opportunities throughout the life cycle of the project. The risk
register cannot be developed and then placed on a shelf. It must be DEFINING THRESHOLDS
made part of the project’s ongoing progress monitoring and reporting The key document output in project risk management is the risk register
with contributions from all action officers. It is important to keep [2]. A sample risk register has been provided in Appendix A – Sample Risk
communication clear and concise so that the risk management objectives Register. Prior to diving into risk identification and analysis, it is important
are not lost. to align to thresholds by which each risk will be ranked. Thresholds
It is important for the project team to recognize that the risk are needed for likelihood, time impact, cost impact, and total impact.
management cycle (risk identification, analysis, response planning, Samples are provided in Figure 1. To associate value to a ranking system,
and monitoring/controlling) will be ongoing throughout the project thresholds of each ranking should be discussed by the stakeholders of
lifecycle. Throughout this lifecycle, the risk register will be the be the the risk workshop to gain alignment to the ranking threshold. These
central document that will keep the focus of this qualitative process. The thresholds are often driven by the risk capacity of each stakeholder or the
document will be accessed frequently for the need for introducing new objectives of the project.
FIGURE 1 Project Risk Ranking Criteria FIGURE 2 Time Horizon Table
Figure 1 provides ranking

criteria for four key elements, all
of which will be ranked from very
low risk to very high risk on a scale
of 1 to 5.
• Likelihood of occurring with at

least a noticeable effect on cost
and/or time.
A. Shown in terms of %
probability
• Time Impact, generally the
impact on the final milestone.
Participants must consider
whether the risk affects
critical path activities or not to
calibrate the time impact.
B. Shown in terms of FIGURE 3 Risk Register Entries From Risk Identification
calendar days, for
stakeholder ease of use
C. Shown in terms of % of contract duration Implementing Qualitative Risk
Note: Only one rating (days or %) will be calculated for time impact
• Cost Impact (D and E), Management at the Project Level
D. Shown in terms of dollars, for stakeholder ease of use
E. Shown in terms of % of contract value RISK IDENTIFICATION
Note: Only one rating ($ or %) will be calculated for time impact Once the team has a common understanding of the language and thresholds
• Total Impact to be used when discussing and managing risk on the project, the next key
F. Score may be calculated by multiplying probability by average of focus is to have the team identify the project risks and opportunities. This can
time and cost impact. Max value in this matrix is 25. take place in the form of a risk workshop or through confidential interviews.
In the authors’ experiences, a combination of both formats is beneficial.
Not all risks are likely to occur at the same time. Assigning a time There may be risks that are unpopular to discuss or even raise in a workshop
horizon to each risk can help prioritize risks to better plan and allocate setting. Examples could be risks that are counter to what a stakeholder may
resources. Time horizon is the forecast of how soon a risk event may occur, want or what may have been reported to management already. These risks
as shown in Figure 2. These thresholds should also be aligned by the generally can be uncovered in confidential interviews and then incorporated
stakeholders and are likely to vary based on the duration of the project. into a workshop later without revealing the original source.
In the sample risk register provided in Appendix A, the impact rating is Risk descriptions should be framed as a statement that includes, “if
calculated as shown in Equation 1. The impact rating score can be used to “X” action happens, it could generate a “Y” risk event, resulting in “Z”
identify the risk with the highest impact to the project. It should be noted consequences,” [2, p. 9]. A brainstorming session is critical during the risk
that depending on the priorities of the stakeholders, one may desire to identification stage so that there can be an open exchange of perspectives.
calculate a separate impact rating for cost only or time only. This would There should be an understanding that there are no “bad ideas” or
be done by simply multiplying the time impact rating by the likelihood “poor contributions”. All perspectives should be collected and can be
rating. In this example the two ratings are being averaged together to get refined later. A critical point here is that each person within the project
an overall prioritization. management team will have a different perspective of the project. As such,
each person brings a valuable lens with which to look at project risk and
should be included in the risk process due to the value that each brings to
the risk management process and to the project delivery objectives. This
EQUATION 1 Total Impact Rating Score must be acknowledged and encouraged for a thorough risk identification
stage to take place.
Figure 3 provides a
sample of the portion
of the risk register
entries that would
be present after the
identification stage.
Common Challenges
of Risk Identification
Risk identification
determines which
risks might affect
the project and thus
should be documented.
Identifying and FIGURE 4 Risk Ratings
assessing risk
are essential components of project management; however, often, have a common understanding of key terms in earlier stages, here it is
smaller projects may not undertake risk assessment in a formal or important to have a common basis for how the impact and likelihood
rigorous manner. Just getting started poses challenges for many project values will be determined. Often, project teams take a “gut check”
teams. Teams must remember that this process is continuous and is approach like throwing darts at a wall while blindfolded. It is better to
to be undertaken regularly with participation from across all project have a set of criteria and thresholds that define how impact and likelihood
stakeholders. Risks need to be clearly defined (realistic, relevant, tangible, rankings are defined as shown in Figure 1, as well as time horizon
and written in an if-then statement). As described in AACE Recommended criteria shown in Figure 2. It is prudent to evaluate both cost impacts
Practice 62R-11, the risks should be clearly defined, so that the precise and schedule impacts for each risk. Figure 4 provides a sample of the
nature of the risk can be understood by all stakeholders. This could lead to additional columns filled out in the risk register for each entry shown in
mitigation and control difficulties. It is hoped that risks are actionable, that Figure 3.
is subject to mitigation since most are threats. Stakeholders can identify
risk through lessons learned, meetings, surveys, historical documents/ Stakeholder Considerations in Risk Identification and Analysis
past projects, and assumptions. Documenting every risk provided and In a perfect world, every member of the project team and all stakeholders
reviewing thoroughly is important because finding out what can go wrong, would be interviewed to gather every potential known risk; but the reality
or right for opportunities, is the most important step. If risks are not is that these individuals are unlikely to have the time nor the budget for an
identified, then they cannot even have a chance of being controlled. At that extensive risk management exercise.
point the project may be plagued with unanticipated events that could A great place to start is a conversation with the project’s owner.
have been avoided or mitigated with advance planning. Understanding the owner’s risk tolerance and key concerns is a good
launching point for the risk register and will frame the scope and
Risk Event vs. Issues extent of the risk discovery efforts. Remember that as risks are gathered,
There needs to be a clear understanding of the difference between a project information should also be gathered about the risk’s probability of
risk event versus an issue. A risk event is an incident or occurrence whose occurrence, the time horizon for the risk, and the impact it would have if it
nature or result could be a threat or opportunity to the outcome of the occurred and is captured in the risk register.
project. A risk condition is a specific identifiable circumstance that might The contractor’s project and field management team are the experts
affect the outcome of the project. [1]. Generally, these risk events are related in delivering the project, and it is imperative that they are involved in risk
to schedule, budget, or quality. If an event actually occurs, it is an issue not a identification. Not only are they familiar with the challenges of installing
risk. If there is a certain level of probability less than 100% that the risk event the work, but they have the pulse of the industry. Most often, the greatest
or condition will occur with some measurable consequence (uncertainty that risks are not those on the project site. Availability of skilled labor, supplier
matters), this is a risk not an issue. For example, if a piece of equipment was capacity, and industry economics, to name a few, often pose the greatest
not delivered and production of the project cannot continue this is an issue. risks to a project. If the contractor’s staff identifies and calibrates risks
The equipment has not been delivered and production has already stalled. independently from the owner’s staff in local workshops or interviews,
The team must react and figure out how to recover the schedule. On the other these parameters should be presented to an integrated workshop and
hand, the team knows the equipment supplier has been late with delivery of blessed by all sides.
material in the past and if equipment is late the project’s production will be The architect/engineer (A/E) of record should know the project like
delayed. This is a risk. The project team can develop mitigation strategies to the back of their hand since they designed the project. The A/E brings
limit the probability of occurrence, perhaps by paying a premium to reduce a valuable point of view to risk identification through their knowledge
the impact to the schedule. Should the equipment be late, i.e., the risk actually of similar projects, lessons learned of what went wrong and what went
occurred, the risk becomes an issue. right. Lessons learned are some of the best inputs to the risk identification
process because they define realized risks, actual impacts, and point to
QUALITATIVE ANALYSIS proper future mitigation strategies.
This step in the risk cycle is focused on quantifying the potential impact There may also be other stakeholders whose input is crucial to
of the risk and the probability the risk will occur. As it was important to project completion. These could include specialist subcontractors, such
as commissioning agents not represented by the EPC prime contract, Management Framework, 2nd Edition, there are four types of responses for
suppliers of equipment or services or regulatory/community groups who threats and opportunities. For threats, or negative risks, the four responses, in
are not necessarily included in the EPC contract and schedule. Their risks order of preference, are: avoid, reduce, transfer, and accept. For opportunities,
may become important and should be included. the four responses are: exploit, share, enhance, and accept [3]. The “Current
Update Notes” column can be used during progress and planning meetings
PRIORITIZATION to keep all stakeholders abreast of the current state of the risk.
At a minimum the two basic components of prioritization are a factor of
the impacts and likelihoods qualified in the previous stage. Further, the
timing of the potential risk should be considered. Priority should be given
to risks identified that could affect the job immediately over risks that will Common Challenges with
not affect the project until further in the future. The total impact rating and
time horizons can be used in concert to filter and prioritize identified risks. Implementation
RESPONSE PLANS One of the main reasons that projects increase in cost and time and decrease
The development of response plans is a critical element to the risk in quality is by not managing risk. Many project teams do not think ahead and
management process. This may not necessarily take place all at one time, look for better solutions. Project managers rely on experience and intuition to
or in a risk workshop, or at all. The initial stage of developing the response plan their work and implementation of risk management is relatively low in
plans should be to identify who needs to do what and by when. Each risk consistency and formality compared to other project management practices.
should be assigned to a risk owner. The risk owner will be responsible This may be due to lack of information and knowledge within the industry.
for developing the response plan. The timing information identified in To overcome these challenges the industry should focus on integrating risk
previous steps can be used to help prioritize when response plans need to management into daily and weekly operational planning, company cultures
be ready. Develop response plans in a proactive manner, well in advance and providing education. Risk management and risk documents that are
of when the potential risk event is expected to occur. Waiting until the created for projects need to be more than preventative planning documents
event occurs to think about a response plan defeats the purpose of the risk that check a box and sit on a shelf.
management process, which is to be proactively planning for and where Another challenge is that the mitigation actions may require changes
possible, mitigating the potential impacts of known risks. The response to contracts, working patterns, and hiring more people. Generally, projects
plan should be able to be incorporated into the execution plan and do not want to expend more money, but many of the mitigation actions
schedule seamlessly before the event occurs. require it. The best project risk implementations are to realistically discuss
Finally, the amount of control or the ability to affect that the project the costs and benefits of mitigation actions so that those actions will be
team has over the potential risk event should be considered. For example, adequately resourced and hence more likely to be effective.
the project team may have very little control over a risk event associated
with weather. With probability that is due to weather, the impact may be
mitigated by prioritizing the building closure to a water-tight condition.
However, a potential risk event, associated with a design element, could be Maturation of Risk
under the team’s control to influence risk mitigation options.
Figure 5 provides a sample of the additional columns filled out in the risk Management Processes
register for each entry shown in Figure 3. The risk response column can be Thus far, this article has addressed risk management processes at a
used to categorize the type of response plan chosen. Per the AACE Total Cost project level. What if a stakeholder’s organization wants to expand their
risk management
processes to an
enterprise level? The
first step is assessing
the current state
of the organization.
Fussell discusses
risk management
capability assessments
(RMCA) and provides
a sample table as
shown in Figure 6 [4].
This or other similar
RMCA models can
be used to evaluate
the current state of
an organization’s risk
management program,
although not limited
to enterprise level risk
FIGURE 5 Response Plans management.
To expand from risk
management on a single project to
enterprise level risk management
programs, there are both physical
and behavioral elements to address.
Physical components include, but
are not limited to:
• established processes to create

consistency across projects,
• templates for various steps of
the program, and
• systems for capturing and
analyzing risks and lessons
learned from multiple projects.
Behavioral components include

but are not limited to:
• consistency in and
accountability to following
established processes, FIGURE 6 Example Risk Management Maturity Model [4, p. 6]
• feedback to historical data,
• feedback to estimating, and Focusing on qualitative risk analysis allows any project to engage no
• contribution to enterprise level systems. matter what level of project data and this process can be implemented by
any project team. During analysis, risks are reviewed against the project
risk criteria in order to the ranking of risk for the project.
Quantitative risk analysis differs in that risks are assigned quantitative
Evolution from Qualitative ratings in order to develop a probabilistic analysis of the project, typically
using Monte Carlo simulation techniques on a project schedule and/or
Analysis at the Project Level cost estimate. This analysis requires well developed schedules and data
to simulate to produce probable project outcomes with specific outcomes
The following two evolutionary steps are mutually exclusive and are such as a particular contingency for schedule completion and project cost
natural next steps to the growth of an organization’s risk management that can be reported at a desired target level of certainty. Identified risks
program. and background uncertainty drive the activity durations and the cost
elements as shown in RP 57R-09 [9].
FROM PROJECT LEVEL TO ENTERPRISE LEVEL RISK Since qualitative risk management is a subjective look by project teams
MANAGEMENT at project risks and quantitative risk management provides an analysis
Evolving from project level risk management to enterprise level risk using the project schedule and budget in a modern Monte Carlo approach
management starts with consistent risk management at the project level. using data collected from the project teams as well, stakeholders can be
Processes must be consistent on individual projects, as well as across more comfortable relying on quantitative risk outcomes of specific project
projects. One of the goals of enterprise level project risk management is to elements. There are numerous tools and techniques available to perform
be able to share lessons learned, risks, and response plans across project quantitative risk analysis:
– to use risk outcomes on one project to prevent risk on another. This
requires the organization to have defined risk categories and a process to • Decision tree analysis
capture project level data from multiple projects into a single depository. • Three-point estimate of background uncertainty
As project level data is collected and aggregated, it can then be analyzed • Identified risks used to drive the quantitative model, hence can be
and used to provide input to the risk identification and analysis steps on prioritized for risk mitigation.
future projects. • Monte Carlo analysis of cost and schedule
FROM QUALITATIVE TO QUANTITATIVE RISK MANAGEMENT As the organization’s risk team develops, they become more familiar
It is important to understand the difference between qualitative risk and comfortable with advancing their risk management knowledge and
analysis and quantitative risk analysis. Qualitative risk analysis prioritizes capabilities. Quantitative risk management is generally lead by a risk
risk using a predefined list of criteria. Risks are scored based on likelihood expert that can focus using the above techniques on high risk or high
of occurrence and their potential impact on project objectives. Qualitative priority risk for individual projects. More can be found on these processes
risk analysis delivers an easy-to-understand risk register for the project in the following AACE International Recommended Practices:
team that can be filtered or focused based on time horizon, score,
probability, or other attributes in the risk matrix. • 42R-08 Risk Analysis and Contingency Determination Using Parametric
Estimating [6]
• 43R-08 Risk Analysis and Contingency Determination Using Parametric 11. AACE International, Recommended Practice No. 66R-11, Selecting
Estimating – Example Models as Applied for the Process Industries [7] Probability Distribution Functions for Use in Cost and Schedule Risk
• 44R-08 Risk Analysis and Contingency Determination Using Expected Simulation Models, Morgantown, WV: AACE International, Latest
Value [8] revision.
• 57R-09 Integrated Cost and Schedule Risk Analysis Using Risk Drivers 12. AACE International, Recommended Practice No. 113R-20, Integrated
and Monte Carlo Simulation of a CPM Model [9] Cost and Schedule Risk Analysis and Contingency Determination using
• 65R-11 Integrated Cost and Schedule Risk Analysis and Contingency Combined Parametric and Expected Value, Morgantown, WV: AACE
Determination Using Expected Value [10] International, Latest revision.
• 66R-11 Selecting Probability Distribution Functions for use in Cost and
Schedule Risk Simulation Models [11]
• 113R-20 Integrated Cost and Schedule Risk Analysis and Contingency ABOUT THE AUTHORS
Determination Using Combined Parametric and Expected Value [12] Jessica M. Colbert, PSP, is with Allan Myers and
can be contacted by sending email to: jessica.
colbert@allanmyers.com
Conclusion
By engaging in some form of risk management, the team is showing
investment in protecting the project outcome from potential harmful
impacts to cost, time, safety, continuity of operations, or quality.
Implementing risk management only increases the chance of project Kimberly Forbes, PSP, is with MBP and can be
success and leads to an organization maturation of risk management contacted by sending email to: kforbes@mbpce.
processes aiding in continue success of future projects. It is definitely in com
the project’s interest to invest time, staff attention, and even mitigation
actions to conduct risk analysis and management.
References
1. AACE International, Recommended Practice No. 10S-90, Cost Appendix A–
Engineering Terminology, Morgantown, WV: AACE International, 2004.
2. AACE International, Recommended Practice No. 62R-11, Risk Sample Risk Register
Assessment: Identification and Qualitative Analysis, Morgantown, WV:
AACE International, Latest revision. (see following page)
3. Stephenson, H.L. Ed., Total Cost Management Framework: An
Integrated Approach to Portfolio, Program and Project Management,
2nd ed., Morgantown, WV: AACE International, Latest revision.
4. Fussell, L. “Risk Management Capability Assessment,” AACE
International, Morgantown, WV, 2015.
5. AACE International, Recommended Practice No. 41R-08,
Understanding Estimate Ranging, Morgantown, WV: AACE
International, Latest revision.
6. AACE International, Recommended Practice No. 42R-08, Risk
Analysis and Contingency Determination Using Parametric Estimating,
Morgantown, WV: AACE International, Latest revision.
7. AACE International, Recommended Practice No. 43R-08, Risk Analysis
and Contingency Determination Using Parametric Estimating –
Example Models as Applied for the Process Industries, Morgantown,
WV: AACE International, Latest revision.
8. AACE International, Recommended Practice No. 44R-08, Risk Analysis
and Contingency Determination Using Expected Value, Morgantown,
WV: AACE International, Latest revision.
9. AACE International, Recommended Practice No. 57R-09, Integrated
Cost and Schedule Risk Analysis Using Monte Carlo Simulation of a
CPM Model, Morgantown, WV: AACE International, Latest revision.
10. AACE International, Recommended Practice No. 65R-11, Integrated
Cost and Schedule Risk Analysis and Contingency Determination Using
Expected Value, Morgantown: AACE International, Latest revision.
16
Appendix A–
Sample Risk Register
J U LY/AU G U ST 2023
The Importance of Owner/
Contractor Transparency
Across All Contract Types
BY STEPHEN L. CABANO
ABSTRACT
Many owner organizations use lump sum contracting to deemphasize the need for diligent transparency
of project cost, schedule, risk, resources, etc. They reduce oversight and superficially transfer responsibility
for successful project execution - with the attitude that ‘it’s the contractor’s job’ to control the project. This
approach eliminates attention to detail with the historically accepted premise that the owner may be limited
in access to performance data that can be expected from a fixed-price contractor. History has proven that
the above approach is often unsuccessful and that owners can only partially transfer project risk to the
contractor. At the end of the day, owners may end up with surprise cost/schedule impacts, contractors who
are suddenly struggling financially, and the realization that it’s too late for effective corrective or mitigation
actions. Project control transparency is critically important for any selected contracting strategy, and the
extent to which contractors are willing to share trends is often mischaracterized. While reporting on internal
rates, overhead buildups and full-cost impacts may be seen as somewhat restrictive, many contractors are
willing to share worker hour unit rates to provide adequate visibility for productivity measurement, as well as
overall project staffing. It has also been observed recently that many contractors are willing to provide added/
improved transparency in an overall trend toward contracting partnerships versus the previously experienced
adversarial relationships. The detailed information required will vary, but the owner’s primary goal should be to
maintain visibility and an understanding of reliable, accurate project status that will facilitate an effective owner/
contractor relationship. This article will include an outline of a reasonable data request which, when correctly
communicated, will encourage a win-win situation. This article was first presented at the 2022 AACE International
Conference & Expo as OWN-3820.
execution challenges, changing business climates, pricing fluctuation,
Introduction: Industry product, and supply chain availability changes, and others. These risks
have always been an issue when planning and executing capital projects
Perspective but never have they been as impactful as during the past two years. The
pandemic has accentuated these risks and dealt capital project planning
For the focus of this article, issues across the heavy industrial sector and execution another low blow.
of capital projects will be addressed. Many of these same issues are The industry continues to try to address this risk management aspect
encountered in other industry sectors but this article will focus on oil of projects through effective contracting, but many have swung the
refining, petrochemical, pharmaceutical, power, material processing, and pendulum too far either one way or the other. Some contracts hold the
other related process industry projects. execution contractor 100 percent liable for all project risks, other contracts
The capital project world, like all other aspects of life today, has never hold the owner responsible for all project performance issues, while others
seen a more chaotic environment than has been experienced over the are silent when it comes to unplanned issues (worst case option). Owner
past two years. The project world before the pandemic was fraught with project teams need to evaluate, during the front-end planning process,
major project cost overruns and schedule delays brought on, in part, by a who has the best resources available, owner or contractor, to manage
lack of communication between owners, contractors and suppliers. This the risk and develop the contract strategy appropriately, to address the
lack of communication initiates poor business case definition from the selected risk management approach.
owner and a constant changing of the overall scope of work driven by The industry is not standing still while these above-mentioned
poor stakeholder management. This created an environment of an evolving issues occur. Many professional societies including AACE International,
scope, changing business drivers and poor execution planning across all the Construction Industry Institute™ (CII), the Project Management
project planning and execution entities. Institute® (PMI), Engineering and Construction Contracting® (ECC)
This was not the only issue the project world needed to address; the Association, Construction Users Roundtable (CURT), and others, are
industry continues to see a “brain drain” across the industry, accelerated attempting to address these issues through research, best practice
recently by the “great resignation” across all aspects of the capital project deployment, innovative collaboration techniques, etc., which has
environment. The construction industry was particularly hit hard, losing stemmed the tide of project deterioration. The pandemic also impacted
over 1.1 million jobs in March and April 2020 alone, according to a report the ability of these society/organizations to continue to positively impact
by construction IT services provider ConstructConnect [1]. This has and the industry. Companies have been extremely cash conscious throughout
continues to create a less experienced, less skilled, and less knowledgeable the pandemic and have limited training, society participation, and other
capital project planning and delivery resource pool across owners, project improvement initiatives, as these are easy cost cutting measures.
contractors, and suppliers [2]. This resource issue is in big part the cause However, some corporate spending is finally returning to address the
of the above-mentioned stakeholder management issue due to a lack of outlined issues.
understanding of what this project management responsibility is all about.
This continues to cause major productivity issues, as well as cost increases
and schedule delays across our capital project portfolios. The pandemic
has also fed this trend, with many project representatives asked to “stand Historical Project Results
down” during the epidemic which forced them to consider different
career opportunities. Many have not returned across the engineering, The project environment has not been conducive to delivering projects
procurement, and construction disciplines. on time and within budget. On the other hand, much success has been
Another aspect of the industry that continues to decline is risk realized in project safety. Figure 1 and Figure 2 chart the historical safety
management. Capital projects are inherently risky, not only from a health performance of the industry vs. safety performance of those organizations
and safety aspect but from the business and project execution aspects, as that focused on this issue through various safety improvement initiatives
well. The world has always been full of numerous project planning and lead by CII [3].
FIGURE 1 CII Members

Reported TRIR (RIR)
Rate, 1989-2019
FIGURE 2 CII
Members Reported
DART (LWCIR) Rate,
1989-2019
Although still not at zero incident and zero fatalities, it shows that the These results continue to deteriorate the business value of owner
industry has improved greatly in this area over the past 30 years. This does capital investments and impact the contractor community through massive
not come without an impact on both capital cost and execution schedule. project losses, divestitures, mergers, and bankruptcies. There are many
It takes money and time from both owner and contractor organizations contractors who will not bid on heavy industrial projects or at least will not
to be safe, and it need not negatively affect this important aspect of work bid on fixed price/lump sum on these projects. This is quite detrimental
while trying to improve on other project delivery issues. Safety is always for the owner community as roughly 85% to 95% of all work executed on
the number one project objective. capital projects is executed through contracted services. If the contractor
There have been numerous studies in recent years that have evaluated community continues to exit this aspect of the industry, owners will
the results of major capital projects over the years. Figure 3 and Figure struggle to execute their capital spending plan and drive value through
4 depict several studies conducted by a number of industry-leading their planned capital expenditures.
benchmarking and project evaluation organizations, such as CII [4]: The recent pandemic was also impactful to the project delivery
industry. The site requirements such as physical distancing, temperature
testing, and other types of testing, quarantining requirements, etc., all had
an impact on the industry. Many of these health regulations will remain
after the pandemic is over. Some of these provided positive results, as
the individual workers often became more efficient at their duties as they
had more space to carry out their work, they were more focused on their
assignment, and they experienced higher individual productivity. This was
counteracted by the limit to the number of people allowed on the jobsite
and the quarantining required after positive testing.
An environment needs to be created that provides a win-win-win
relationship between the owner, supplier, and contracting communities.
Although there are many aspects of this improvement initiative, this article
will address the role the contracting strategy, and the cost engineering
interface between all parties, has in being a part of this improvement program.
FIGURE 3 Evaluation of Project Results
Alternative Contracting
Strategies
A review of a few contracting alternatives will assist with understanding
the terminology used in this article. The following are several aspects of
the division of work that need to be addressed:
• Engineering
• Studies (ground water, soil conditions, etc.)
• Procurement
• Expediting
• Permitting
FIGURE 4 Recent Mega Project Results • Construction Management
• Construction These are just a few of the multitude of compensation methodologies.
• Inspection Owners are required to develop a contracting plan that integrates both the
• Commissioning and Start Up division of the work and the compensation methodology to best suit the
• Project Controls business and project objectives. This contracting plan often is depicted
• Document Management using what the industry calls a “contracting quilt.” See Figure 5:
• Information Technology/Communication This tool allows the overall contract approach for a project to be
• Safety/Security depicted in one diagram. It outlines the division of the work, and within
• Health Management each work segment, what type of contract is planned, including its
• Etc. compensation strategy. Note that any area not addressed by a contract
remains the owner’s responsibility (blue in the example). One additional
This is not an all-inclusive list but does provide a realization that note, the example does not define the level of transparency required by the
someone needs to be assigned to all these tasks across the life of a contractor on aspects such as planned hours, cost, schedule vs actual hours,
project. As previously mentioned, 85% to 95% of these tasks are typically cost, and schedule. This will be defined going forward in this article.
contracted services.
Next, when looking at compensation methodologies, the following are
the typical compensation schemes used to pay for work delivered with a
brief description of each of the approaches: Industry Perception of
Lump Sum – Payment for a defined scope of work is made on a fixed Transparency Requirements
price basis. The contractor bids on the lump sum offering, if selected,
payments are typically made on agreed to milestone achievements. Over the years, the industry has developed a perception of what data is
The contractor’s profit is built into the lump sum amount. required to be communicated from the contractor to the owner based on
the compensation methodology chosen. The perception exists in both the
Reimbursable Cost – Payments are made based on hours used. The baseline data as well as the information transferred at each of the defined
contractor bids on a billable rate structure and is paid on number of reporting periods. This is a fallacy derived over the years based on a
hours spent. The contractor’s profit is typically built into the hourly rate. lack of collaborative project delivery. Contractors wanted to hide project
aspects, such as profit and productivity data, and owners wanted to hand
Reimbursable Cost w/fixed Fee – Payments are made based on hours overall accountability and responsibility of performance to the contractors.
used. The contractor bids on a billable rate structure and is paid on This was addressed earlier when discussing risk management.
number of hours spent. A contractor’s profit is typically agreed to as a The trend in the industry is that a contracting approach using a
fixed amount and allocated based on milestones achieved. reimbursable cost methodology, or a variation of, would require full
Unit Price – The scope

of work is defined in
units - the contractor
bids on price per unit. If
selected, the contractor
is paid for the number
of units completed.
Contractor profits are
typically built into the
unit rates.
Incentives – Numerous
incentive programs
are available that are
structured to provide
contractors with
financial incentives
for targeted project
objectives such as
schedule improvements.
These are above and
beyond the base
compensation structure.
Incentives are typically
structured as lump sum
payments.
FIGURE 5 Contracting Quilt
transparency as that is how the contractor gets paid. The basis of the contract with no recourse for recovery. The benefit lies in a collaborative approach
would be a list of reimbursable hourly rates and a plan on the number of hours to resolving project issues. The owner and contractor can work together to
needed to execute each of the defined scopes of work, typically broken down address either contractor issues such as resource constraints or experience
by a detailed level of the work breakdown structure (WBS) called the work limitations or owner issues around poor scope definition or changing project
package. An example of the WBS and a work package are included in Figure 6. drivers. All these issues can be extremely detrimental to project success but
The contractor then provides the hours worked vs. the hours planned, often are addressed by only one party with limited success.
and this defines productivity. The contractor is paid for all hours worked, Full transparency will also allow the collective project team, owner,
no matter how productive they are. This technique is not used on lump and contractor, to address issues and act as one team. This will also lower
sum contracts. the owner’s anxiety regarding project status and allow them to address
In a lump sum contracting approach, the perception is that the issues more effectively and communicate with senior management with a
contractor is not required to provide the same level of transparency as higher level of confidence.
outlined under the reimbursable methodology because it is perceived that
owner does not have the right to this information. This is incorrect. The
owner has the right to require any level of transparency desired, as long
as it is stated in the request for proposal (RFP) documentation and agreed Detriments of Full Transparency
to in the contract. Contractors have the right to not bid if they are not
comfortable with the approach the owner is requiring. So, there can also be a downside to this level of project data transparency.
So why would an owner desire to have the same level of transparency The owner needs to recognize that this data transfer and communication
across any of the contracting compensation schemes? The following does not come for free. The contractors will build this requirement into
sections of this article will outline some of the reasons why. their proposed pricing as it will take a bit more time to develop reports
and analysis as defined. The owner also needs to have the appropriate
level of resources on their team to support the data flow. The owner’s
role in project controls is that of assurance and validation, as defined in
Benefits of Full Transparency Pathfinder’s AACE® International paper “The Owner’s Project Control
Function,” dated January 2011 [5]. If this is not done effectively it can cause
Full transparency is achieved on all reimbursable contracts as that is the project delays and cost increases by overassessment of the data.
basis for payment. This also requires additional oversight from the owner The application of this transparency needs to be a collaborative effort and
to validate the hours and assess the contractor’s trends and forecasts. not an oversight or punitive role when any issue does not meet baselines.
This is a great practice if the owner has the resources to support these Contractors need to be permitted some leeway against baselines. These
tasks. This provides the data driven details from the contractors and tolerances need to be established as part of the project control plan portion
their assessment of progress and productivity along with the owner’s of the overall project execution plan (PEP). The typical content of the PEP
independent assessments for a true check and balance of the data. is outlined in Figure 7 (see following page), as well as the table of contents
This technique can also be used in a lump sum environment to assure the of a high-quality project controls plan (See Figure 8 on page 23) as defined
owner that project status and progress are as stated by the contractor. Trust by AACE International Recommended Practice 60R-10: Developing the Project
yet verify. If not applied, there could be misalignment of project status and Controls Plan [6]:
productivity issues and the owner could get blindsided late in the project
FIGURE 6 Sample
Work Breakdown
Structure (WBS) and
Work Package
during the bidding process and other factors
become more important. If contractors do
not believe they can build the facility at the
cost defined, they just refuse to bid.
The objective of either of the above
examples, or others, is to bring the
contractors, suppliers, and the owners
closer together in a more collaborative
relationship. This is being addressed today
through the application of advanced work
packaging (AWP). The AWP approach is
to bring construction knowledge up to
the front-end planning and engineering
efforts in an attempt to make the execution
phase of work more efficient. “Construction
driven engineering” is the phrase being
applied to this methodology.
This approach has proven to provide
much more accurate early project cost
estimates, better schedules earlier in the
process and better owner decision making.
Today’s electronic tools, aligned with the
digitization of all project data, will provide
an excellent platform for more innovative,
real time project controls information.
Project status will be derived from the
digital twin on actual work performed
against work planned and all tied to the
FIGURE 7 PEP Format and Content same work packages as is defined on the
schedule. Individual implementation work
packages (IWPs) will be tied to procurement work packages (PWPs)
Both the owner and contractor teams need to be aligned on both the and construction work areas (CWAs) to develop segments of work that
PEP, as well as the project controls plan prior to contract execution. are more defined and easier to implement. All this information would be
integrated into the engineering work packages (EWPs) so engineering can
design to accommodate and efficient construction environment.
Much more work is to be done in the AWP arena, but project control
What the Future Holds will need to change to accommodate the evolution of the AWP approach.
There are many changes being considered in contracting. Alternative

contracting schemes such as integrated project delivery (IPD) and
operating system 2.0 (OS2)[4] are being defined and tested by many of Conclusion
the previously mentioned societies. These collaborative methodologies rely
on the alignment of goals, wherein the contractor does better if the owner Project performance data is as important now as ever, but it will take
does better. “Goal congruence” is the term used. on a different look and functionality as the industry evolves. For the
Examples of this might be that the contractor is made aware of the foreseeable future, the project control plan will still be required to assure
owner’s business case and the contractor buys into the owner’s business there is alignment across all project parties on what the data transparency
proposition. The contractors’ profits would be based on a small portion requirements will be for a given project. Traditional contracting strategies and
of the owner’s asset profitability. If the contractor can develop the asset in project control requirements can and do work. Some of the more innovative
an efficient manner and provide one that is efficient to run and maintain, contracting and project control approaches are proving to deliver more data
the overall profitability for the owner and thus, the contractor, increases/ transparency and better team alignment of project status and issue resolution.
improves. A win-win for both parties. If the project and resultant asset are
less efficient, the profits decrease/diminish for both parties. This is not
suitable for all projects, but an interesting approach less.
Another example would be to take the capital cost out of the equation. References
The owner develops a business case that is substantiated through the
financial analysis including the capital cost and operating cost of the asset. 1. Jones, Kendall. State of the Construction Industry: One Year into
The owner bids on the project at a defined cost. Contractors do not bid on the Pandemic, Construct Connect Online Blog (https://www.
cost, they bid on the quality of their team, their execution approach, the constructconnect.com/blog/state-of-the-construction-industry-one-
efficiency of the final operating asset, etc. This takes capital costs off the table year-into-the-pandemic), Cincinnati, OH, April 2021.
FIGURE 8 Project Controls Plan - Table of Contents
2. Aschman, Allison, Why Capital Project Systems Succeed or Fail, ABOUT THE AUTHOR
Independent Project Analysis Newsletter, Volume 10, Issue 3, Ashburn, Stephen L. Cabano is with Pathfinder, LLC
VA, September 2018. and can be contacted by sending email to:
3. Construction Industry Institute, Improving Safety Performance slcabano@pathfinderinc.com
through Operational Excellence, CII Research Team 317, Construction
Industry Institute, Austin, TX, August 2018, CII Annual Conference.
4. Construction Industry Institute, Refining the Capital Projects Business
Model: Operating System 2.0, Construction Industry Institute, Austin,
TX, Latest revision.
5. Cabano, Stephen L, Paul G. Williams, and Robert C. Mathias.
The Owner’s Project Control Function, 2011 AACE International
Transactions, OWN.503.
6. AACE International, Recommended Practice No. 60R-10, Developing
the Project Controls Plan, Morgantown, WV: AACE International,
Latest revision.
Implementation of a
BY ERIC VYSKOCIL
ABSTRACT
The importance of an integrated database for project historical data to measure project
performance is recognized by the recent AACE International Recommended Practice
114R-20. A historical database allows project teams to create standardized reports on
key performance indicators, trend metrics over time, and analyze the data to observe
common trends in project performance. There are many challenges to overcome in the
development of an integrated database: establishing data input standards; maintaining
the data quality; establishing and maintaining the necessary data security; resolving
database errors in a timely manner; and developing effective metrics. This article will
focus on the key challenges while implementing an integrated project data warehouse
at the Ontario Power Generation (OPG) facility, and some of the benefits that have been
observed. The development of this data warehouse began in 2015 as part of a mega-
project and was extended to all OPG projects in 2019. This article was first presented at
the 2022 AACE International Conference & Expo as DSAA-3798.
Introduction
An integrated, historical project database is an important tool in
allowing organizations to make data driven decisions, calculate key
performance indicators, trend metrics over time, and analyze the data
to observe common trends in project performance. The importance of
such a tool to manage and store data is reflected by the recent release
of AACE International Recommended Practice 114R-20 [1], as well by a
number of recent AACE technical papers concerning integrated reporting
and analytics [2] [3] [4]. The experience of the teams at Ontario Power
Generation (OPG) in developing its historical project database reflect the
lessons that are presented in the AACE International Recommended Practice
114R-20 [1], emphasizing its usefulness and relevance.
Either directly or through wholly owned subsidiaries, OPG is
responsible for 66 hydroelectric stations, two nuclear stations, one
biomass station, one dual-fueled oil and gas station, four combined cycle FIGURE 1 Illustration of OPG’s data architecture and relationships (prior to
gas turbine stations and one solar facility in the province of Ontario. At any 2015) between key source systems. Note that there are no relationships
given time, there are hundreds of projects ongoing to complete upgrades, between relevant source systems.
repairs, decommission old facilities or build new facilities. Responsibility
for management of these projects has been split between different
organizations, called business units, each of which had its own processes,
requirements, and tools to support their activities. An illustration of the
relevant data systems in existence prior to the Darlington refurbishment
project and their lack of relationships is shown in Figure 1.
The development of an integrated project historical database at OPG began
in 2015, as part of the planning phase for the refurbishment of the Darlington
Nuclear Generating Station. The Darlington Nuclear Generating Station is
a CANDU heavy water moderated, heavy water cooled nuclear generating
station that has been in operation since the early 1990s. The Darlington
refurbishment project is a program of over 500 individual projects totaling
$12.8 billion that will be executed between 2016 and 2026 to replace critical
components, allowing the station to continue operations until 2055. [5]
The project historical database, internally called the Integrated
DataBase (IDB), was originally intended to be used to provide a single
source of truth for project data related to that work. By 2019, the scope
of the database expanded to include all projects executed by OPG and FIGURE 2 Illustration of the early IDB architecture and relationships to
include nearly all data areas relevant to project management. The IDB key source systems with notes about the frequency and types of updates
now consists of multiple databases that together integrate data from a and data history method.
large number of internal OPG systems, including Oracle’s Primavera P6
for schedule data, Hexagon’s EcoSys for cost data, and TIBCO’s EBX for
additional project information, risks, actions, and issues. engineering, and other project information. This included purchasing
In order to develop this infrastructure, it has been necessary to off-the-shelf systems such as Primavera P6 and EcoSys, hiring a third-party
overcome a number of challenges to standardize processes for project vendor to develop two custom web applications for project and risk data
management, streamline and modernize the tools being used, and and development of in-house OPG Microsoft Access databases for major
continuously monitor to ensure the data quality was sufficient for reporting scoping and engineering data. All users were required to follow common
and analysis purposes. The resulting setup has allowed OPG to conduct procedures, applying the same standards with respect to updates, and
data analysis to validate the existing metrics and implement new ones that OPG led teams were established to monitor data quality.
are being used by project teams to inform their decision-making process. Development of the IDB by a third-party vendor contracted by OPG
began in parallel, with extract, transform, and load (ETL) procedures
being created to download the data out of the relevant source systems.
As shown in Figure 2, for each data source, different data processing
Data Infrastructure and retention rules were applied. Some of the updates were completely
automated and some data systems, like the cost system, required
Development significant manual intervention.
Instead of relying on external vendors to develop and maintain
EARLY DEVELOPMENT automated reports, based on the IDB data for the project teams, it was
As part of the Darlington refurbishment project, certain tools were decided to establish an internal group of OPG project services staff to
established to be used by all vendors to input schedule, cost, risks, perform this development, calling it the data infrastructure and reporting
team. This team would work closely with the project execution teams ENTERPRISE EXPANSION
in order to develop and maintain key project and portfolio metrics Starting in 2018, led by the newly established project management
and visualizations. This arrangement was expected to provide a closer office, OPG embarked on an initiative to apply enterprise-wide common
collaboration with project teams and quicker turnaround on change standards and processes to all projects executed for the organization,
requests or new development than would be possible with an external including the use of standard processes and tools that applies to all
vendor. Some of these visualizations included data quality metrics, projects at OPG. During the consultation process with the various
S-curves similar to those in AACE International Recommended Practice 55R- business units, it was found that there were very few consistent processes
09 [6] and variance metrics such as those referenced in AACE International or tools in use across the organization. For example, large programs, such
Recommended Practice 86R-14 [7]. as the Darlington refurbishment program and projects executed at both
These automatic reports were created using SQL Server Reporting Darlington and Pickering nuclear generating stations, used Primavera P6,
Services (SSRS), hosted on SharePoint, and were the only way that while other parts of the organization used Microsoft Project.
individuals outside of the data infrastructure and reporting team could In order for OPG to achieve project excellence and the ability to report
access the data. Access to reports was restricted by data source and on project performance across the enterprise, rules were established such
managed through role-based access with individuals being granted that all business units were to use common tools. Based on the experience
access by the data owners. As a result of the different data extraction and of the Darlington refurbishment projects, Primavera P6 and EcoSys were
processing rules, there were no consolidated reports relating schedule, selected for schedule and cost data, respectively. As IDB was already
risk, and other information together. connected to these systems, going forward it would store historical data for
all projects in OPG.
ESTABLISHING A DATA MART The scope, cost, and complexity of projects executed by OPG
As the need for project reporting increased, along with the expectation for varies from relatively straightforward updates to office layouts, to more
ease of access to the relevant data, the data infrastructure and reporting complicated projects with significant nuclear safety considerations. As
team expanded their responsibility beyond just the development of a result, it was understood that a one-size-fits-all approach would either
automated reports to include streamlining the data infrastructure, apply too much or too little rigor to the projects. Therefore, a scalable
reducing the number of data systems involved, and expanding the user model for project management requirements, based on the overall project
base for the automated reports. cost and complexity, was implemented. The higher the cost or complexity,
In order to improve query performance, centralize business rules, and the greater the requirements for project teams to populate data into
allow for changes to be applied and modified in an expedited manner, a relevant source systems. At the lowest level, a project would only be
data mart was established for OPG to manage internally without the need required to maintain the estimate at completion and key milestones.
to involve third party vendors. This data mart pre-processes data using To ensure that all business units were able to access the data for their
overnight ETLs from the original IDB database and applied centralized projects, the automated reports were re-built to allow customizations
business rules to improve report performance and align reporting logic. across the business units while still requiring certain standards, such as
These business rules are managed by the data infrastructure and reporting key performance indicator metric calculations. To allow for the creation
team in collaboration with the relevant data and process owners. of a portal that helped users more readily find the data they needed, the
At this point, as shown in Figure 3, the development of the data reports were migrated off SharePoint and into a custom web application
infrastructure included connections to most key project data systems. The developed and maintained by the data infrastructure and reporting team.
data was modified by business rules and summarized in the data mart for Users’ experiences could also be tailored to their role and previous activity
automated reports to generate visualizations, reports, and dashboards, to on the site.
support project and management teams. As the data for a project was now While the overall data infrastructure was not significantly changed
centrally stored, it became possible to create dashboards integrating multiple initially, it was recognized that there would be modifications necessary to
aspects of a project’s performance for the Darlington refurbishment. standardize data across the different source systems, account for the data
security expectations across business units, provide new functionality
for project teams to input relevant project data, and digitize the existing
project management processes. This led the data infrastructure and
reporting team to investigate options for a master data management
(MDM) tool to achieve this.
IMPLEMENTING A MASTER DATA MANAGEMENT (MDM)

SOLUTION
To fill the need for a master data management solution for the new project
management organization at OPG, the data infrastructure and reporting
team selected EBX, now owned by TIBCO. This software had already
been in use in a limited way by OPG, so there were individuals who had
the necessary development experience. It was expected that the team
would use EBX to progressively replace existing custom web applications,
Microsoft Access databases, and paper processes with digital forms and
workflows. Internally, to distinguish these new developments from other
EBX solutions at OPG, the project teams began to refer to the collection of
FIGURE 3 Illustration of the IDB architecture and relationships to key functionality as Enterprise Project Management Extensions (ePMX).
source system after creation of an OPG managed data mart.
At this time, the system managing project risks, actions, issues, and
lessons learned needed significant upgrades to sustain it. Presented with
the options to continue to sustain the existing platform and make the
necessary upgrades or migrate to a new platform, the choice was made to
migrate to ePMX. In addition to the impact on a number of downstream
systems and migration of existing data, this change also required a
significant amount of effort working with stakeholders and training new
users. The end result was a platform that better aligned with other source
systems, had built-in data quality checks, and provided more functionality
and customizability than the original system.
Figure 4 illustrates the data infrastructure after this development
work related to the MDM solution was complete, with key source systems
having been built into EBX and the legacy systems having been deprecated.
ONGOING DEVELOPMENTS
The ePMX system continues to expand to include new data areas related
to project management, including change control, capital in-service, gate
progression, and business case information. Additional source systems
are being implemented and connected to the IDB, such as Copperleaf’s
C55 for asset management information and InEight for estimating data.
FIGURE 4 Illustration of the IDB architecture and relationships to Integrations between key source systems are also being established to
key source systems after implementation of the MDM solution and improve data quality and ensure alignment on common data elements.
consolidation of key data source systems. The current state of the IDB and related source systems is illustrated in
Figure 5.
Leveraging the Data

Warehouse
The growing project historical data in the database has allowed OPG to
undertake new analyses that would not otherwise be possible. These
analyses have provided further justification for the reliability and usage of
existing metrics and has led to enhancements of other automated metrics.
ACCURACY OF NEAR-TERM SPENDING FORECASTS

A key metric that was developed early on, and widely adopted by the
organization, was the forecast accuracy factor. This metric provides project
oversight teams with a better understanding of the accuracy of their near-
FIGURE 5 Illustration of the IDB data architecture and relationships to key term cost forecasts. This is calculated as shown in Equation 1 where is
source systems currently implemented at OPG. the forecast spending made in period j for the period i and is the actual
spending in the period i.
The Access database, managing the engineering data, was one of the
first pieces of functionality to be replicated in ePMX and subsequently
retired. As the user base for this tool was limited and there were very few
downstream systems, the transition was easily completed. The database Equation 1
tables were re-built, existing data was migrated out of the access database,
and the existing tools were disabled. By detailed analysis of longer-term trends, it was possible to identify
Soon after, the custom web application managing project information broader trends about the accuracy of the forecasts being made by project
was also migrated to ePMX, with the existing data migrated into the new teams. As shown in Figure 6, it was determined that while project teams
platform and additional functionality added. Similar to the engineering could reliably forecast spending in the next month, this accuracy was
data, there were a relatively small number of users of this web application, considerably diminished when forecasting beyond one month. As a
but there were significant downstream impacts to consider. The key result, the forecasting reliability of longer term and overall lifecycle
in this transition was to minimize the number of changes required in project costs lacked sufficient basis for predictable project performance.
any downstream systems, automated reports, and other tools. When These results were used to inform changes to the metrics being used
completed, centralizing this project information would form the to encourage project teams to focus on creating more reliable forecasts
foundation for a large number of modifications to come. beyond one month.
COST PERFORMANCE INDEX STABILITY
The cost performance index (CPI) is a standard
metric in project management [8] and has
been identified as having significant predictive
power in some studies [9]. As the project nears
completion it becomes more difficult for the
project team to substantially improve the CPI,
meaning at a certain point in the project the CPI
becomes predictive of the final performance of
the project.
It was therefore decided to investigate the
relationship between the CPI at completion and
the CPI at various points in the project using
the data available for OPG projects. At this time,
OPG only had approximately 50 projects at
completion with sufficient historical cost data.
FIGURE 6 Box and whiskers graph showing the forecast accuracy factor for a subset of projects and Despite the relatively low sample size, as shown
months at OPG. The median forecast accuracy factor is shown by a black line, the boxes show the in Figure 7, it was found that there was a strong
middle two quartiles of values, and the whiskers represent the range of the middle 68% of values. correlation between the CPI at the project mid-
point and the CPI at completion.
The results of this analysis were used to convince project
teams of the fundamental value and predictive nature of the
CPI metric. Due to limitations regarding data availability,
OPG decided to accumulate more data before continuing
this analysis.
INDEPENDENT ESTIMATE AT COMPLETION

The AACE International Recommended Practice 80R-13
recommends that the projects estimate at completion (EAC)
be validated to ensure that it is reasonable [10]. One of the
options presented is the independent estimate at completion
(IEAC), which presents a range of values for comparison
against the EAC prepared by the project team. These values
are calculated by using a set of mathematical equations that
extrapolate past performance.
After internal discussions and endorsement of OPG
project teams, IEAC calculations were implemented in an
automated report that matched Table 1 from the AACE
International Recommended Practice 80R-13 [10], shown in
FIGURE 7 Visualization of the relationship between the CPI at project mid-point and the
Figure 8. This report allows oversight teams an opportunity
CPI at project completion for select OPG projects. Equation for linear fit and R2 value
to focus attention on specific projects where the EAC
calculated by Microsoft Excel.
significantly deviates from the IEAC range and confirm that
the forecasts are reasonable.
These equations were further leveraged into an automated
visualization that compared the EAC trend against the IEAC
range over time, as shown in Figure 9. A review of historical
data identified several cases of forecast optimism by the
project teams that would have been accurately predicted
by the IEAC range. This has encouraged OPG that tracking
this metric provides value as an early warning indicator of
potential future cost variance.
FIGURE 8 Visualization of the table from the automated report implementing the
Independent EAC range using trial data.
Lessons Learned in
Data Warehouse
Development
The following are key lessons learned in
OPG’s data warehouse development. Many
of the lessons that have been learned are also
reflected in AACE International Recommended
Practice 114R-20, under Section 1.3.3 Principles of
Database Development and Management [1].
IDENTIFY A KEY DECISION MAKER

Assembling and consulting with the correct
stakeholders is an important consideration
to developing the right technical solutions.
However, the need to build consensus can FIGURE 9 Visualization of an example of EAC optimism for a project detected by the IEAC range. Also
sometimes override important technical and included is the Budget at Completion (BAC), BAC including contingency and the percent complete.
process considerations and cause a solution to
be selected that is not sustainable, overly complex, or does not provide project teams along with database and report development best practices. Over
sufficient value to the organization. Therefore, it is important to identify a time, an emphasis was placed on ensuring that the data infrastructure and
key decision maker who is empowered to provide strategic direction and, if reporting team remained familiar with the relevant business processes.
necessary, align the stakeholders to a standard approach. This investment is not insignificant, but improving the business
The development of the centralized project controls office at OPG understanding of technical staff has led to improvements that have
provided the necessary organizational structure to empower a single achieved commensurate business value. For example, project teams
individual to make key decisions about processes and tools. Previously, each at OPG are required to periodically present their progress at monthly
business unit managed projects with independent tools, processes, and meetings. To help facilitate these meetings, automated websites and reports
metrics, thus obtaining concurrence on even small issues proved difficult. were developed containing all relevant project information summarizing
This proved essential in resolving disagreements between business the monthly progress. Members of the development team observed the
units on the nuances behind metric calculations, such as CPI, due to monthly meetings to continuously develop their business knowledge and
earning assumptions about level of effort activities [11]. Turnover in understand how project teams used the tools. Observations were made
management led to multiple changes in direction as OPG implemented that project teams had a common practice to manually extract visuals
different versions of the calculation. Beyond the cost of rework, project and metrics from the website and reports into a PowerPoint presentation.
teams also became confused as the CPI calculation changed and became Knowing this, the team proposed and implemented a standardized
more complex. This was resolved when all relevant stakeholders were PowerPoint export for each project. This has allowed project teams to
assembled, the issue was discussed, and a final direction by the key minimize manual effort on each project’s monthly review, while also
decision maker was documented and implemented across all systems. providing standard presentation packages for ease of consumption.
CREATE A DEDICATED DEVELOPMENT TEAM PUT BUSINESS PROCESS BEFORE PROGRAMMING

As stated in AACE International Recommended Practice 114R-20, database Achieving early wins is important to build support for the development
development projects are not a spare time endeavor [1]. A centralized activities and demonstrate the benefits of the overall project historical
and dedicated team can apply standards to the development approach, database. However, when development activities begin before a business
evolve with the business needs, and minimize third-party involvement process has been implemented, the results are less likely to be satisfactory
that can delay progress. Optimum results require this team to have a set of and there is a high risk of not providing a return on investment.
resources with complimentary technical skills and business knowledge. Prior to the development of the project historical database at OPG,
Business knowledge is a crucial element to any development team for key decisions were made to standardize the work breakdown structure
a project historical database. When defining requirements, stakeholders (WBS) and establish naming conventions for work packages in the cost
are not always sufficiently familiar with available technical capabilities and and scheduling systems. All projects were assigned program, bundle,
pure technical staff can be unaware of the challenges project teams face. A sub-bundle, unit, contract number, and project number and as shown in
development team with a proper understanding of the business can develop Figure 10, work package activities were assigned unique 10-digit numbers
solutions that meet the project teams’ needs instead of just their requests. composed of the project number, scope category, control account number,
The original database development team at OPG was created as a mix of and work package number. Although ensuring data quality remained a
individuals with IT backgrounds and familiarity with database development challenge for some time, this WBS provided the foundation to define how
and administration, alongside those with project management experience both the database ingests and processes the data and allows it to be presented
inside and outside of the organization. The team readily shared business and in a standardized way across all projects. Had these decisions been made
technical knowledge, maturing the knowledge set of the team to a standard after the database had been designed and implemented, it would have led
level. This aligned key business terminology, metrics, and expectations of to a significant amount of re-work to obtain the desired result.
PLAN FOR CHANGE
It is inevitable that the data infrastructure
will evolve in ways both small and large.
The systems should be flexible enough to
handle these changes without requiring large
investments of effort.
As the usage of Primavera P6 expanded at
OPG, it was identified that a single instance
was no longer sufficient to handle the amount
of data that was being generated. Since
multiple instances of P6 would be maintained
separately, it was not possible to guarantee
that the two instances would retain identical
data dictionaries. OPG decided to engage an
external vendor to create parallel integration
databases to handle the data imports from
both instances of P6 and consolidate the
data in the OPG managed data mart. One
advantage of this approach was that if it
FIGURE 10 Standardized nomenclature for work package activities in P6 and EcoSys. became necessary to create more instances of
P6, it would be relatively straightforward to
Data Source ETL Schedule ETL Trigger Change Tracking expand this model to integrate all schedule
Schedule Nightly Automatic Snapshots (daily, weekly, monthly) data together consistently.
Separating the technical responsibilities
Cost Monthly Manual Snapshots (monthly)
this way was only possible because of
Risk Nightly Automatic Delta records the earlier decisions to define division of
Engineering Nightly Automatic Delta records responsibilities of the integrated database
Project Nightly Automatic None into a data mart that is managed by the OPG
data infrastructure and reporting team, and
FIGURE 11 ETL and change tracking methodology implemented early in IDB development across a database that is managed by a third party.
key data sources The final infrastructure implemented is
illustrated in Figure 12.
ENSURE EFFORTS ADD VALUE

START WITH THE END IN MIND Changes must be comprehensively evaluated in order to ensure that the
Some of the most difficult challenges to overcome could have been most new functionality adds long-term value to the overall organization. The
easily resolved early in the development of the integrated database. goal is to create tools that provide long lasting value to the organization,
Understanding the end goal of what is intended to be achieved will help without requiring substantial changes over time. These kinds of decisions
plan development and identify challenges that need to be overcome to cannot be made by the development team alone but must be done through
meet that goal. consultation with stakeholders supported by relevant data.
Early in the development of the integrated database, due to a lack of
resources to develop an overall data infrastructure strategy, OPG failed
to establish common requirements across different source systems for
ETL development. As shown in Figure 11, some data sources used full
snapshots to track changes over time, while others tracked changes
through delta records and relied on effective and expiry date fields, and
some sources did not track changes at all.
Furthermore, due to the invoicing cycle the cost month end was set to
be the second week of the following month but the schedule month end was
aligned the last Friday of the fiscal month. This led to project teams being
required to meet different month ends for different systems, significantly
complicating the process for aggregating data for month end reporting.
As part of continuous improvement efforts, these issues have been
ameliorated and going forward, as additional systems are integrated into
ePMX, these considerations are taken into account. However, changing
an existing system to align to a standard requires significantly more effort
than it would if it had been incorporated into the original design. FIGURE 12 Detailed relationship between different instances of P6 and
databases in IDB. Potential new instances of P6 would be added in
parallel as shown.
FIGURE 13 Visualizations from the report usage metrics tool used by the OPG data infrastructure and reporting team, illustrating report penetration
by user groups and overall report usage.
FIGURE 14 (AT RIGHT) Automated report usage

trend for a report developed that meets the
original value assessment.
FIGURE 15 (BELOW RIGHT) Automated report

usage trend for a report developed that
exceeds the original value assessment by
expanding the user base.
The data infrastructure and reporting

team at OPG assembles relevant stakeholders
from different business units to review
all requested modifications and approve
them. It was expected that for automated
reports, requesters would provide a business
justification commensurate with the effort
required to complete the work. Where
possible, requesters would be provided access
to the data to help them develop mockups of
the desired visuals to assist in this process.
Automated reports were also created to
internally track report usage, identify key
report users, and measure report processing
wait times. Some of these visualizations are
shown in Figure 13. These metrics, together
with feedback from the stakeholder team were
used to help prioritize report modifications
and improvements. Post implementation
reviews were also performed to determine
if the modifications fulfilled their original
business value statement.
After initial deployment of a new tool,
usage should remain consistent or increase
over time, as shown in Figure 14, ensuring
that original value assessment is fulfilled.
Periodically reminding users groups about the
existence of particular tools and their purpose
can help drive increased usage, maximizing
years and continues to this day, with new data
sources and integrations being created. Along
the way, numerous challenges involving data,
people, technology, and processes have been
encountered and overcome in order to create
a tool that integrates and stores data from
numerous source systems.
Establishing the appropriate organization
and processes to support the project historical
database has been found to be equally
important as the technical development.
Identifying a key decision maker provides the
ability to make strategic decisions and provide
unified direction. A dedicated development team,
including individuals with relevant technical
and business knowledge, allows solutions to
be developed and implemented in an efficient
manner. Without standards and processes for
teams to follow when using sources systems, the
resulting data will not provide any value.
Based on OPG’s experience, developing
FIGURE 16 Automated report usage trend for a report developed that failed to meet the original a vision for the technical changes, both large
value assessment. and small, is key to long-term sustainability
and success. With a strategic vision, changes
to the database can be planned to align with
the benefit from these tools at very little cost, as shown in the usage trend existing structures, thus reducing sustainment costs. Spending the
in Figure 15. The overall goal is to avoid situations, as shown in Figure 16, necessary front-end effort and time to anticipate future changes and
where a tool may receive initial praise, but usage will quickly drop off if the develop flexible solutions can avoid significant amounts of rework in
tool fails to provide the benefit originally defined by the requesters. the future. Interrogating the value of requested changes allows the
development teams to focus on the things that provides the most value to
USE DATA TO TELL STORIES the organization.
Once sufficient data has been collected, teams must move from just As OPG’s data infrastructure has matured, it enabled the data
calculating metrics to performing more detailed analysis of the data and infrastructure and reporting team to tackle more difficult questions and
producing proposals for changes that should occur. A quote from Willard perform more detailed analysis. This has led to the finding that providing
C. Brinton is that “as the cathedral is to its foundation so is an effective descriptive metrics is insufficient to support project excellence. The team
presentation of facts to the data” [12] [13]. must further develop innovative predictive and prescriptive metrics that
During an OPG project portfolio review meeting, it was once identified help ensure OPG is able to successfully complete its projects safely, on
that more than half of the milestones in one group of projects were forecast time, on budget and with quality, every time.
to be late to their baseline plan. When one of the attendees questioned
what impact this would have on overall project performance, there was
no definitive or obvious answer. It is important to keep in mind when
developing metrics that the goal is not for project teams to improve their References
metric performance, but to improve their project performance, and a
metrics usefulness is directly tied to its ability to predict whether the 1. AACE International, Recommended Practice No. 114R-20, Project
project will meet its objectives. Historical Database Development, Morgantown: AACE International,
As a result, the data infrastructure and reporting team re-directed Latest revision.
significant attention into developing data analytics that supported the 2. Ayers, A. and S. W. Bomba, “Once Upon a Project: Using Data to Tell
use of specific metrics, some of which are shown in the previous sections. the Story,” in TCMA-3054, AACE International Transactions, AACE
In addition, the team reviewed best practices from other organizations, Conference & Expo, Virtual, 2021.
including the AACE recommended practices and conferences and looked 3. Ayers, A., S.W. Bomba and L. El Didi, “Data Analytics to Drive
for ways to implement these ideas at OPG. The goal is to move from simply Reporting Insights for Timely Decisions and Improved Business
describing the data to predicting future results, and ultimately prescribing Performance,” in TCMA-3674, AACE International Transactions,
the actions necessary to project teams. AACE Conference & Expo, Los Angeles, 2019.
4. Marcus, G. “The Impact of AI-Driven Project Management on an
Organization’s Decision-Making Culture,” in TCMA-3674, AACE
International Transactions, AACE Conference & Expo, Virtual, 2021.
Conclusion 5. Prince, O., B. Sabrina and A. Pedro, “Continued Operation of
The development of an integrated, historical project database at OPG Darlington Station: An Impact Analysis on Ontario’s Economy,” The
has been a step-by-step process that has been occurring over the last six Conference Board of Canada, Ottawa, 2016.
6. AACE International, Recommended Practice No. 55R-09, Analyzing 12. Brinton, W.C. Graphic Methods for Presenting Facts, The Engineering
S-Curves, Morgantown: AACE International, Latest revision. Magazine Company, 1914.
7. AACE International, Recommended Practice No. 86R-14, Variance 13. Few, S. Show Me the Numbers, 2nd ed., Burlingame, CA: Analytics
Analysis and Reporting, Morgantown: AACE International, Latest Press, 2021.
revision.
8. A Guide to the Project Management Body of Knowledge (PMBOK Guide), ABOUT THE AUTHOR
6th ed., Newton Square, PA: Project Management Institute, 2017. Eric Vyskocil is with Ontario Power
9. Christensen, D.S. and S. Heise, “Cost Performance Index Stability,” Generation and can be contacted by sending
National Contract Management Journal, vol. 25, 1993. email to: e.vyskocil@opg.com
10. AACE International, Recommended Practice No. 80R-13, Estimate at
Completion (EAC), Morgantown, WV: AACE International, Latest
revision.
11. Lukas, J.A. “How to make earned value work on your project,” in PMI®
Global Congress 2012 - North America, Vancouver, British Columbia,
Canada, Newtown Square, PA: Project Management Institute.
UAL
Call
ANN
Submissio
For Deadline
Aug. 31, 2 :
n
023
Papers
IS NOW OPEN!
We are now accepting abstracts
for our 2024 Conference & Expo
Signia by Hilton Atlanta
Atlanta, Georgia, USA
June 16 - 18, 2024
Ready to To present a paper at an AACE Conference & Expo, a

100 to 200 word abstract of the proposed paper must be
submit your submitted on the abstract form with the call for papers.
abstract? If your abstract is accepted, your full manuscript/paper

will be due no later than January 31.
SUBMIT PAPERS HERE! LEARN MORE!
ENERGIZE
Your
Career
Becoming a Certified Cost Professional
is a proven way to enhance your value to
employers and clients by providing an impartial
endorsement of your knowledge and expertise.
Earning your Certified Cost Professional (CCP) Spark change, become a

Certification shows your professional commitment
and ability to your peers, supervisors, and clients,
and may give you that competitive “edge” when
CERTIFIED COST
being considered for a promotion or future
opportunity. In addition, AACE International salary
PROFESSIONAL
surveys reveal that Certified Cost Professionals For more information
earn more at all steps throughout their career than visit web.aacei.org
those without certification.

AACE Journal - Cost Engineering July - Aug 2023

Uploaded by

Copyright:

Available Formats

AACE Journal - Cost Engineering July - Aug 2023

Uploaded by

Document Information

Original Description:

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

AACE Journal - Cost Engineering July - Aug 2023

Uploaded by

Copyright:

Available Formats

T H E J O U R N A L O F A A C E I N T E R N AT I O N A L | T H E A U T H O R I T Y F O R T O TA L C O S T M A N A G E M E N T

J ULY / AUG U ST 2023

The Importance of Owner/Contractor

CORE is the Association for the Advancement

INTEGRITY Demonstrated by WHY WE EXIST

Learn more, management, asset management, and dispute resolution. Members

or join today! such as construction, manufacturing, engineering, infrastructure,

EXPLORE OUR WEBSITE

17 The Importance of Owner/

ALSO IN THIS ISSUE

3 AACE International Board of Directors

17 3 Cost Engineering Journal Information

24 For additional industry news and updates,

1 Time 6 Technical Development

PAST PRESIDENT GRAPHIC DESIGN Little Fish Design Company

Policy Concerning Published Columns, Features, and Articles

for presentation at the 2024 TECHNICAL ARTICLE 1

which abstracts and/or technical TECHNICAL ARTICLE 2

YOU MAY NOT KNOW THAT . . .

NEW AUTHOR PRESENTS FIRST ARTICLE

BRIDGING THE GAP

Figure 1 provides ranking

• Likelihood of occurring with at

• established processes to create

Behavioral components include

FIGURE 1 CII Members

FIGURE 3 Evaluation of Project Results

Unit Price – The scope

There are many changes being considered in contracting. Alternative

IMPLEMENTING A MASTER DATA MANAGEMENT (MDM)

Leveraging the Data

ACCURACY OF NEAR-TERM SPENDING FORECASTS

INDEPENDENT ESTIMATE AT COMPLETION

IDENTIFY A KEY DECISION MAKER

CREATE A DEDICATED DEVELOPMENT TEAM PUT BUSINESS PROCESS BEFORE PROGRAMMING

ENSURE EFFORTS ADD VALUE

FIGURE 14 (AT RIGHT) Automated report usage

FIGURE 15 (BELOW RIGHT) Automated report

The data infrastructure and reporting

Ready to To present a paper at an AACE Conference & Expo, a

abstract? If your abstract is accepted, your full manuscript/paper

SUBMIT PAPERS HERE! LEARN MORE!

Earning your Certified Cost Professional (CCP) Spark change, become a

You might also like

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.