LESSON 6

ESTABLISHING A QUALITY ASSURANCE AND IMPROVEMENT PROGRAM

OVERVIEW

Standard 1300 – Quality Assurance and Improvement Program states, “The chief audit executive must develop
and maintain a quality assurance and improvement program that covers all aspects of the internal audit activity.”
The QAIP should encompass all aspects of operating and managing the internal audit activity—including consulting
engagements—as found in the mandatory elements of the IPPF. It may also be beneficial for the QAIP to consider
best practices in the internal audit profession. Implementation Guide 1300 states, “The QAIP is designed
to enable an evaluation of the internal audit activity’s conformance with the International Standards for
the Professional Practice of Internal Auditing (Standards) and whether internal auditors apply the IIA’s Code of
Ethics.” Trough conformance with the Standards and the Code of Ethics, the internal audit activity also
achieves alignment with the Definition of Internal Auditing and the Core Principles.

Establishing a Quality Assurance and Improvement Program

The QAIP must include ongoing and periodic internal assessments, and external assessments by a qualified
independent assessor or assessment team from outside the organization. Quality should be built into the way the
activity conducts its business—through its internal audit methodology, policies and procedures, and human
resource practices. Building quality into a process is essential to validate and continuously improve the internal
audit activity, demonstrating value as defined by stakeholders.

Delivering quality requires a systematic and disciplined approach as professionals. Quality does not just happen; it
is the combination of the right people, the right systems, and a commitment to excellence. Building an effective
QAIP is similar to establishing a total quality management program where products and services are analyzed to
verify that they meet stake-holder expectations, operations are evaluated to determine their efficiency and
effectiveness, and practices are assessed to confirm their conformance to standards. Maintaining an effective
QAIP also requires leaders who are responsible for setting the proper tone in support of quality and continuous
improvement.

Using key concepts of quality as a foundation in establishing QAIP, the internal audit activity should consider all
mandatory and recommended guidance element of the PPF that support:

• Conformance with the standards and the Code of Ethics. It is further understood that through
conformance with the standards and the Code of Ethics, the internal audit activity also achieves
alignment with other mandatory elements of the IPPF.

• Stakeholder satisfaction defined by expected and preferred internal audit deliv-erables that produce
value for the organization.

• Operational effectiveness achieved by building quality “into” internal auditprocesses. Preventing

mistakes is generally less costly than correcting mistakes.

• Continuous improvement of internal audit activities accomplished throughquality initiatives identified

during the quality assessment process.

• Management commitment to provide resources and tools necessary for a QAIP to succeed.
Participation is expected by all members of the internal audit activity.
For the internal audit profession, it is important to ensure that internal audit activities glob-ally
maintain the highest possible standards of service delivery to the organizations they support. The
IIA established the IPPF to guide the internal audit profession, and the mandatory elements of
the IPPF—supported by recommended guidance—are the foundation for developing an internal
audit activity’s QAIP
THE QAIP FRAMEWORK
Standard 1300 – Quality Assurance and Improvement Program states that the CAE must develop
and maintain a QAIP that covers all aspects of the internal audit activity. Common elements of
all QAIPs include:
• A scope that includes all aspects of the internal audit activity.
• An evaluation of conformance with the Standards and the Code of Ethics
• An appraisal of the efficiency and effectiveness of the internal audit activity.
• The identification of opportunities for continuous improvement.
• Involvement by the board in oversight of the QAIP.
A framework is oftentimes used to describe the complete environment for developing and impl
ementing the QAIP. An example of such a framework, consisting of Governance, Professional
Practice, and Communication, is shown in the figure below.

Continuous Improvement
of IA Processes

Reporting
and Follow Up
Internal Audit Activity
Findings, Observations
and
improvement of QAIP

Recommendations
Continuous
Periodic Self-Assessment
Professional Practice

Ongoing Monitoring

External Assessment
communication
Governance

Quality Assessment

Quality Assurance Over

Entire IA Activities

Figure 1 - THE QAIP FRAMEWORK

Quality Assurance and Improvement Program Framework
To construct a QAIP framework, the internal audit activity universe must be considered. This
universe must include the IPPF, and may include the legal requirements of the specific country
and/or industry where the activity is operating, stakeholder expectations, use of third party
subject matter experts, co-source partners for internal audit services, and the size and structure
of the overall organization. Implementation Guides for the 1300 series of the Standards provide
more detail and insight.

INTERNAL ASSESSMENTS
Two key elements of the quality assessment process comprise the internal assessment portion
of the internal audit activity’s QAIP:
1. Ongoing monitoring, and
2. Periodic self-assessments.

Ongoing Monitoring
What is important to remember is that a QAIP must be built into the processes of the internal
audit activity and not onto the way the activity conducts its business. The most obvious internal
method for continuously assessing quality is management oversight of internal audit work.
Adequate supervision from the beginning through the end of the engagements is a fundamental
element of a QAIP.

The Deming Cycle (or Plan-Do-Check-Act cycle) provides a possible structure in establishing the
QAIP. The steps in the Deming Cycle are as follows:
1. Plan - means establishing expectations for operating a process to meet specific objectives,
goals, or deliverables.
2. Do - means executing the process and collecting data for analysis and follow-up in the
Check and Act steps of the cycle.
3. Check - is the step where actual results are compared to expected outcomes and
differences are analyzed.
4. Act - is where feedback is provided to the operators of the process to reinforce
expectations established in the previous Plan step. It is in this step that improvements to
the process are identified and implemented.

Applying the Deming Cycle to the ongoing monitoring portion of the QAIP might look like the
figure below:
 PLAN
• Establish department standards for
engagements
• Create checklists (planning, meeting
agenda, and engagement closeout
procedures)
• Design templates (risk control matrix, test
plans, and process documentation.
• Develop tools (data mining and sampling
techniques)
• Design formats (issues/findings and reports)

ACT DO
• Provide coaching and take corrective • Plan, perform, and report
action. engagements.
• Reinforce standards through • Use checklists, templates, tools, and
communication and training. formats.
• Revise checklists, templates, tools, and • Collect data on engagement process
formats as needed performance.
• Verify department standards are

CHECK
• Verify department standards are met or
exceeded.
• Confirm use of checklists, templates,
tools, and formats.
• Document supervisory review.
• Record, report, and analyze metrics

Figure 2 - Ongoing Monitoring

The ongoing monitoring element of the QAIP would primarily address conformance with the
following Standards since they are intended to address quality on an audit-by-audit basis and
relate primarily to engagement activities:

• 2200: Engagement Planning

• 2300: Performing the Engagement
• 2400: Communicating Results
• 2500: Monitoring Progress
To this end, ongoing monitoring applies to all assurance and consulting assignments and should
achieve the objectives described in Standard 2340 – Engagement Supervision, which states,
“Engagements must be properly supervised to ensure objectives are achieved, quality is assured,
and staff is developed.” Tis standard also requires that appropriate evidence of supervision is
documented and retained. Tis documentation provides assurance that ongoing monitoring is
incorporated into the routine policies and practices used to manage the internal audit activity. In
other words, a quality review must be performed for each engagement. This review provides an
opportunity for ongoing evaluation, coaching, and feedback for each auditor assigned to the
engagement.
As noted in Implementation Guide 1311 – Internal Assessments, ongoing monitoring mechanisms
may include:
• Checklists or automation tools to provide assurance on internal auditors’ compliance with
established practices and procedures and to ensure consistency in the application of
performance standards.

• Feedback from internal audit clients and other stakeholders regarding the efficiency and
effectiveness of the internal audit team. Feedback may be solicited immediately following
the engagement or on a periodic basis (e.g., semiannually or annually) via survey tools or
conversations between the CAE and management.

• Staff and engagement key performance indicators (KPIs), such as the number of certified
internal auditors (CIAs) on staff, their years of experience in internal auditing, the number
of continuing professional development hours they earned during the year, timeliness of
engagements, and stakeholder satisfaction.

• Other measurements that may be valuable in determining the efficiency and effectiveness
of the internal audit activity. Measures of project budgets, time-keeping systems, and
audit plan completion may help to determine whether the appropriate amount of time is
spent on all aspects of the audit engagement. Budget-to-actual variance can also be a
valuable measurement to determine the efficiency and effectiveness of the internal audit
activity
Results of ongoing monitoring must be reported to the board or the audit committee at least
annually. The adequacy and effectiveness of the ongoing monitoring portion of the QAIP should
also be evaluated as part of periodic self-assessments

PERIODIC SELF ASSESSMENTS

Implementation Guide 1311 – Internal Assessments states, “Periodic self-assessments have a
different focus than ongoing monitoring in that they generally provide a more holistic,
comprehensive review of the Standards and the internal audit activity. In contrast, ongoing
monitoring is generally focused on reviews conducted at the engagement level. Additionally,
periodic self-assessments address conformance with every standard, whereas ongoing
monitoring frequently is more focused on the performance standards at the engagement level.
”The internal audit activity conducts periodic self-assessments to validate its continued
conformance with the Standards and Code of Ethics.
Through conformance with the Standards and Code of Ethics, the internal audit activity also
achieves alignment with the Definition of Internal Auditing and the Core Principles. In addition,
periodic self-assessments may evaluate:
• The quality and supervision of work performed.
• The adequacy and appropriateness of internal audit policies and procedures.
• The ways in which the internal audit activity adds value.
 • The achievement of KPIs.
• The degree to which stakeholder expectations are met.

The QAIP should document and define a systematic and disciplined approach to the periodic self-
assessment process, which may incorporate programs that may be implemented along this
aspect. Successful internal audit practice is for periodic self-assessment to be performed at least
annually. This provides an annual basis for assurance that the internal audit activity continues to
operate in a manner consistent with requirements of the Standards and the Code of Ethics. This
is especially important during periods of change in the Standards or in the organization.

Many internal audit activities find it valuable to review and update their infrastructure,
methodology, and processes on an annual basis as a component of their periodic self-assessment
to ensure these elements are current with the requirements of the Standards.

This annual periodic self-assessment process provides the board with assurance that the internal
audit activity maintains the standard of performance that is required by the IIA.
Recommendations for improvement should be tracked by a follow-up report, and the results of
which listed at each board meeting. The periodic self-assessment element of the QAIP would
primarily address conformance with the following series of Standards:
• 1000: Purpose, Authority, and Responsibility
• 1100: Independence and Objectivity
• 1200: Proficiency and Due Professional Care
• 1300: Quality Assurance and Improvement Program
• 2000: Managing the Internal Audit Activity
• 2100: Nature of Work
• 2200: Engagement Planning
• 2300: Performing the Engagement
• 2400: Communicating Results
• 2500: Monitoring Progress
• 2600: Communicating the Acceptance of Risks
• Code of Ethics
The periodic self-assessment should also assess results of ongoing monitoring. Applying the
Deming Cycle to these additional elements of the QAIP might look like figure 3:
.
 PLAN
• Create internal audit activity charter.
• Adopt The IIA’s Code of Ethics.
• Establish internal audit activity structure,
policies, and procedures.
• Agree on value-added activities with
stakeholders.
• Establish appropriate measures to track
value-added activities.
• Define relevant quality metrics.

ACT DO
• Assess and report on conformance with • Perform annual audit planning.
IPPFmandatory guidance. • Schedule engagements and assign staff.
• Identify gaps in conformance and develop • Hire, train, and develop staff.
road mapsto close gaps. • Perform ongoing monitoring of
• Revise internal audit activity structure, engagements.
policies, andprocedures as needed • Communicate and meet with
stakeholders

CHECK
• Conduct surveys and interviews with
stakeholders to confirm value is
delivered.
• Review a sample of engagement to
assure ongoing monitoring is
effective.
• Record, report, and analyze metrics.
• Assess internal audit activity
structure, policies, and procedures
conformance withIPPF mandatory

Figure 3 – Periodic Self-Assessment

ASSESSMENT, EVALUATION, AND REPORTING

Establishing an internal assessment process, both ongoing monitoring and periodic self-
assessments, coupled with the reporting of KPIs, culminates in an evaluation of the internal audit
activity’s QAIP, with results reported to appropriate stakeholders.

Two questions the CAE should consider when performing a QAIP evaluation are:
1. Is the evaluation to be a comprehensive or partial assessment of the QAIP and the internal
audit activity?
2. What rating scale will be used to support a conclusion regarding the QAIP and the internal
audit activity’s conformance with the Standards and the Codeof Ethics?
Answering the first question will depend on the design of the internal audit activity’s QAIP and
the level of resources devoted to the internal assessment process. As noted previously, a
successful internal audit practice is to perform annual self-assessments; the Standards do not
specifically state a frequency. Some CAEs may view internal self-assessments as action taken
during years when an external assessment is not performed. Certain parts of the QAIP maybe
evaluated every year, while other portions may be evaluated less frequently.
The second question is not specifically addressed in the Standards, as they do not prescribe an
assessment scale; however, the Standards do require the degree of conformance with the
Standards and the Code of Ethics be assessed.

An evaluation summary framework that contains conformance criteria linked with the Standards
and the Code of Ethics are available, which CAEs can use to assess the conformance with these
mandatory elements of the IPPF. Assessment scale of Generally Conforms, Partially Conforms,
and Does Not Conform are used in the evaluation.

Standard 1320 – Reporting on the Quality Assurance and Improvement Program states, “The
chief audit executive must communicate the results of the quality assurance and improvement
program to senior management and the board.” Therefore, conclusions arising from the internal
assessments of the internal audit activity’s conformance with the Standards and the Code of
Ethics should be provided to key stakeholders as described by the standard. Results of ongoing
monitoring of performance must be reported annually. Successful internal audit practice also
suggests that the results of periodic self-assessment be reported at least annually.

CONTINUOUS IMPROVEMENT
While the primary focus of the QAIP must be on evaluating conformance with the Standards and
the Code of Ethics, real value for the internal audit activity is derived from a focus on continuous
improvement. Internal audit activities that have embedded the concept of continuous
improvement into their operating culture and QAIP go beyond conformance with the Standards
and the Code of Ethics and realize many additional benefits, including:
• Positioning the internal audit activity for success within the organization.
• Becoming more forward-looking in approach and experiencing greater alignment with the
organization’s strategies and objectives.
• Greater adaptability in implementing incremental internal audit process changes,
resulting in greater responsiveness to emerging stakeholder expectations.
• Enhanced internal audit productivity following the elimination of non-value-added
activities.
• Improved internal audit staff morale resulting from a focus on process improvements
where all ideas are welcome.

The concept of continuous improvement highlights the dynamic nature of establishing and
maintaining an effective QAIP. Changing stakeholder priorities, shifting organizational strategies,
and fluctuating environmental factors all contribute to this dynamic. CAEs should not expect
“perfect” or “absolute” conformance with the Standards and the Code of Ethics, particularly for
internal audit activities that are just beginning to establish their QAIP. Conscientious periodic self-
assessments will highlight areas where the internal audit activity can get stronger.

Internal audit activities with mature QAIPs may have moved beyond general conformance, but
they are still focused on continuous improvement of their processes. Organizations use a gap
analysis—comparing current performance with desired future performance—to develop plans
or road maps to achieve process improvements. Reports documenting the internal assessment
process of a QAIP—both ongoing monitoring and periodic self-assessment—should contain
summaries of continuous improvement efforts within the internal audit activity. Tis focus on
continuous improvement within the QAIP assures key stakeholders of the internal audit activity’s
commitment to quality

E XTERNAL ASSESSMENTS
So far, steps to building an effective QAIP have been outlined, focusing on the internal
assessment process—ongoing monitoring and periodic self-assessment. External assessments
are also an element of the QAIP as prescribed by the Standards; however, the Standards only
require an external assessment to occur at least once every five years. Internal assessment
components of the QAIP should be continuously active between external assessments,
establishing the foundation of a successful internal audit activity.

The primary link between a QAIP and an external assessment is the reporting process originating
from the QAIP. For a QAIP to be deemed effective, CAEs should expect external assessors to
affirm what the CAE is measuring in regard to conformance with the Standards and the Code of
Ethics through the periodic self-assessment process and reporting of results to key stakeholders.
The CAE’s report of the periodic self-assessment may be used as a basis for assessment by an
external assessor.

A secondary link between a QAIP and an external assessment is the documentation maintained
by the CAE as evidence of an effective QAIP. This includes charters, policies, procedures, metrics,
audit reports, annual audit plans, engagement workpapers, audit committee minutes, staff
training records, etc. External assessors will want to examine relevant documentation that
describes key elements of the QAIP.

The decision to schedule an external assessment often results from the CAE’s requirement to
perform an external assessment every five years. The CAE might consider other factors when
determining specific timing and scope for this review:

1. Does the CAE believe that the internal audit activity generally conforms with the
Standards and the Code of Ethics?
2. Is the documentation describing the QAIP comprehensive and complete?
3. Has feedback from key stakeholders been incorporated into the QAIP?
4. Have discussions with the board established additional expectations related to
operational or strategic goals?

As noted in Standard 1312 – External Assessments, CAEs can choose from two methodologies for
external assessments. The first approach is a full external assessment, and the second approach
is an independent, external validation of the CAE’s self-assessment of the internal audit activity.
Both approaches—full external assessment and independent, external validation—require that
they be conducted by a qualified, independent assessor or assessment team from outside the
organization. The qualified, independent assessor or assessment team must demonstrate
competence in two areas: the professional practice of internal auditing and the external
assessment process.

Several factors may influence the CAE’s decision in selecting an appropriate external assessment
method to review the internal audit activity’s QAIP. This is an area where the board might take
an active role in oversight of the QAIP as suggested in the Standards.
INSTRUCTION FOR THE REPORTERS

1. Talk among yourselves as to who gets to report what.

2. Try to find additional materials on the internet since we do not have a textbook.
3. An assessment (short quiz – 15 minutes) may be conducted after the reports.
4. Reporters will be rated by the audience.