0% found this document useful (0 votes)
48 views

Harnessinggenerativeai 1

Uploaded by

hưng bùi
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
0% found this document useful (0 votes)
48 views

Harnessinggenerativeai 1

Uploaded by

hưng bùi
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 22

Solving the riddle:

Harnessing Generative AI
for internal audit activities
Executive summary
Organizations around the world are scrambling to solve
the riddle of artificial intelligence (AI). The introduction of
easy-to-use generative AI (GenAI) models, such as ChatGPT,
DALL-E, and Microsoft Copilot, created an unprecedented
level of rapid adoption. The lure of improved efficiency,
productivity, and competitiveness promised by such tools
is too enticing to pass up. Effectively integrating AI into
operations and strategies is the goal. If done right, it can
deliver transformative results to any organization.
The benefits of effectively applying GenAI The Internal Audit Foundation (the
across all operations are unmistakable Foundation) and Wolters Kluwer TeamMate
as organizations leverage the powerful (TeamMate) have collaborated on a
tool’s ability to quickly analyze data, research project that explores the use
revolutionize customer engagement, and of GenAI in internal auditing. This report
automate repetitive processes. Meanwhile, presents the findings of this research,
its drawbacks and limitations are detailing how GenAI is utilized at the
becoming clear. individual, organizational, and internal
audit function levels. Additionally,
For internal auditors, the AI challenge it provides information on GenAI’s
is twofold in that they must not only applications across various internal audit
understand and help guide the use of AI activities and its governance.
within their organizations, but also leverage
the technology to benefit and enhance the
work they do. As with any riddle, the secret Research
to its solution involves applying ingenuity,
curiosity, and resourcefulness. Methodology
“It’s not like we’re looking at AI as The Foundation and TeamMate employed
a replacement of humans,” said Jim a survey aimed at providing an up-to-
Pelletier, a senior product manager for date snapshot of the use of GenAI within
Wolters Kluwer TeamMate. “We’re looking organizations and internal audit functions.
at it as an augmentation to help humans The survey targeted internal audit leaders
be better, to be more efficient, to be more around the world, and its findings provide
productive, to be better at things that valuable insight into how internal audit
you do day-to-day. The collaboration or is currently leveraging this revolutionary
interaction between human intelligence technology as its influence over modern
and AI leads to the best results. “ business strategies grows. The survey,
which took place in May 2024, received a
total of 924 responses.

2
Contents Executive summary

Part I: How GenAI is transforming business 4


2

Understanding GenAI’s strengths and weaknesses 5


AI Resources 5

Part II: Updating the GenAI picture 6


How GenAI is being used 7
• Individual knowledge and use of GenAI 7
GenAI and internal audit are inevitable partners 8
• Where internal audit currently fits in 8
• AI use by internal audit 8
GenAI lacks governance 9
Prompt writing best practices 10

Part III: Use cases 11


Use case matrices 12
• Planning phase 12
• Fieldwork phase 13
• Reporting phase 14
• Follow-up phase 15

Part IV: Next steps 16


Caution is not an excuse for inaction 16
Gen-AI limitations and pitfalls 17

Conclusion 18

Appendix: survey demographics 19

About The Institute of Internal Auditors and the Internal Audit Foundation 20

About Wolters Kluwer TeamMate 21

3
Part I:

How GenAI is
transforming business
The introduction of OpenAI’s ChatGPT, • I nsights Into Consumer Behavior
a publicly available and user-intuitive Capturing data on customer behaviors
generative pre-trained transformer, in the and preferences is fundamental to
latter part of 2022 marked a momentous today’s e-commerce platforms. Knowing
event in the ongoing digital revolution. Its what makes online consumers most likely
record-setting user base has been widely to fill their electronic shopping carts
reported. Today, more than half (55%) of is retail gold. As with KPIs, leveraging
organizations already report limited or customer data to boost sales depends
aggressive adoption of AI into business on human interpretation and lead
operations, according to one industry generation. Missteps can easily lead
report.1 What’s more, AI market value is to customer burnout and churn. With
predicted to more than double from $184 AI analyzing data more efficiently and
billion in 2024 to $415 billion by 2027.2 effectively than humans, customer
outreach can be delivered at the right
The ways GenAI can benefit operations times and reduce churn.
are limited only by the imagination of
executives focused on efficiency and • T
 he Human Factor
productivity. However, three areas have Simply put, AI can perform better than
emerged as those most likely to be humans in certain areas, whether it’s
improved by GenAI: crunching terabytes of data or executing
mind-numbing, repetitive tasks without
• V
 isualization and Monitoring tiring or making mistakes. While there
Developing and measuring Key are legitimate concerns about how this
Performance Indicators (KPIs) is essential could impact some jobs, the trade-off
to modern business. For the most part, between keeping jobs focused on minutia
interpreting just what KPIs are telling us or allowing them to focus on higher-level
is left to humans. However, the benefits of tasks is an easy call.3
KPIs come from accessing the right data
and asking the right questions. GenAI
can improve the process by turning huge
datasets into visualizations that are easily
interpreted and understood. Once in
place, other advanced technology, such
1 Source: CompTIA IT Industry Outlook 2024 report.
as robotic process automation (RPA), can 2 Source: Artificial Intelligence – Worldwide, Statista.com,
accessed June 2024.
be used to monitor key metrics and alert 3 “How Artificial Intelligence Impacts Business,”
managers when anything goes awry. Online Business Blog, Raymond A. Mason School
of Business, William & Mary, April 2024.

4
Understanding GenAI’s strengths and weaknesses
Before blindly embracing GenAI as the LLMs. This creates a ‘jagged frontier.’ Due to a company’s technology stack — AI must be
solution to every problem, organizations the ‘jagged’ nature of the frontier, the same able to flexibly integrate with current and
must have a clear-eyed view of what it can knowledge workflow of tasks can have tasks future systems to draw and feed data into
accomplish effectively and where it may on both sides of the frontier. The future of different areas of the business.”6
fall short. A recently published working understanding how AI impacts work involves
paper from the Harvard School of Business understanding how human interaction with The analysis concludes AI can support
took on the issue of how GenAI affects AI changes depending on where tasks are businesses in three categories: automating
worker productivity and came to a mixed placed on this frontier, and how the frontier processes, analyzing data to gain insights,
conclusion. The authors of Navigating will change over time.”5 and engaging customers and employees.
the Jagged Technological Frontier: Field “Firms that apply AI strategically in these
Experimental Evidence of the Effects of An analysis of AI use in business by online ways have much to gain in terms of
AI on Knowledge Worker Productivity and finance and investment publication productivity, efficiency, and potential cost
Quality tested the premise that three Investopedia identifies integration and savings and growth.”7
factors — the surprising abilities of Large compatibility of AI use as fundamental
Language Models (LLMs), their ability to do factors for success. The Harvard working paper and
real work with virtually no technical skills Investopedia analysis provide valuable
required, and their opacity and unclear “Companies that have successfully insights into what can be referred to as
failure points — create a jagged frontier, implemented AI solutions have viewed the AI/HI line, where AI interacts with
“where tasks that appear to be of similar AI as part of a larger digital strategy, human intelligence (HI). To date, the key
difficulty may either be performed better understanding where and how it can be differentiator is HI’s ability to generate
or worse by humans using AI.”4 instrumentalized to great advantage. novel and creative ideas and provide
This requires considering how it will context, whereas AI is limited to the data
“Some unexpected tasks (like idea integrate with current software and it has been trained on. This factor should
generation) are easy for AIs, while other existing processes — especially how data is remain top of mind as organizations and
tasks that seem to be easy for machines to captured, processed, analyzed, and stored. their internal audit functions consider how
do (like basic math) are challenges for some Another important factor is the structure of to integrate GenAI into their processes.

AI Resources
The following is a list of AI-related resources • A
 rtificial Intelligence Auditing Framework European Union Resources
to explore. These materials offer valuable (IIA members only). This framework AI Innovation Package – This series
• 
insights and information on the latest supports internal auditors’ understanding of documents provides measures to
developments in artificial intelligence use, of risk and identifies best practices and support European startups and SMEs in
regulation, and ethics. internal controls for AI. the development of trustworthy AI that
respects EU values and rules.
IIA Resources Us Government Resources Artificial Intelligence – Questions and
• 
IIA (Institute of Internal Auditors): NIST (National Institute of Standards and Answers – Provides extensive information
• Artificial Intelligence Knowledge Center. Technology, U.S. Department of Commerce): on the EU’s AI Act, who it applies to, and
This online resource provides abundant • Artificial Intelligence Risk Management its goal of addressing AI-related risks to
articles, podcasts, papers, guides, webinars, Framework (AI RMF 1.0). health, safety, and fundamental rights.
events, and tools to better understand AI. • Artificial Intelligence Risk Management
(Please note, some items are exclusive to Framework Playbook. Other Resource
IIA members.) • Artificial Intelligence Risk Management UNESCO (United Nations Educational,
Framework: Generative Artificial Scientific, and Cultural Organization):
Intelligence Profile. Ethics of Artificial Intelligence.

4 “ Navigating the Jagged Technological Frontier: Field Experimental Evidence of the Effects of AI on Knowledge Worker Productivity and Quality,”
Fabrizio Dell’Acqua, et al, Working Paper 24-013, Harvard Business School, Boston, MA, 2024.
5 Ibid.
6 “How AI Is Used in Business,” Elysse Bell, et al, Investopedia, 2024.
7 Ibid.

5
Part II:

Updating the GenAI picture


GenAI’s current state is akin to the opening include providing assurance and advisory
chapters of a mystery where the characters services in areas such as AI strategy,
are identified, and the plot is set. However, governance, model risk, and model
just how the story will unravel is unknown. controls. Domain 2 would focus on AI tools
Several valuable clues are provided by available for internal auditors, such as
the TeamMate and Foundation global GenAI tools, machine learning, and robotic
survey of internal auditors about GenAI process automation.
use. Responses from more than 900
internal auditors about GenAI use in
their organizations and within their audit Domain #1
functions offer encouraging news for
practitioners who have yet to embrace
AI as an
GenAI: You are not alone. Audit Topic
Pelletier, who has spent more than 20 years
in internal auditing, including 10 years
as a thought and guidance leader at The
Institute of Internal Auditors, has a simple Domain #2
but powerful message to practitioners.
AI as a Tool for
“It’s time to get moving,” he said. “Learn Internal Auditors
about it. Take some online courses. Start
using it. If you can’t use it at work yet, start
using it at home so you can learn what
This framework provides a structured
these tools can and can’t do.”
approach to understanding and managing
AI. Organizing AI into two distinct focus
The IIA approaches AI by examining it
areas helps identify:
through various domains that help group
related aspects of AI use within the
• where resources are allocated,
organization and within internal audit.
• where attention is focused,
For example, one option is to create one
• where resources and attention
domain for AI as an audit topic and another
should be directed in the future.
for AI as an auditing tool. Domain 1 would

6
How GenAI is being used
Key data points from the TeamMate and
Foundation global survey suggest there
Figure 1. Individual proficiency and personal use of GenAI
is a significant opportunity for internal
auditors to provide relevant and impactful Q. How would you rate your proficiency and personal use of
influence on adoption and use of GenAI GenAI in your professional role? (n=878).
within their organizations. This should
provide a level of encouragement for 27%
practitioners to act boldly in supporting
this budding business bonanza. 49.4%

Individual knowledge and use of GenAI 17.4%


(Figure 1)
Across every represented role and region, 4.7%
more than 3 in 4 survey respondents rate
themselves as either novices, possessing 1.5%
limited knowledge of GenAI, or as
beginners with minimal knowledge and
application of GenAI, in relation to their Novice: Limited to no knowledge of GenAI and its applications
Beginner: Some basic familiarity but minimal practical application
use and proficiency of GenAI.
Intermediate: Moderate level of experience, with some application
Advanced: Proficient in GenAI, with use to perform work-related tasks
Adoption within the organization (Figure 2)
Expert: Highly skilled, with extensive experience leading use cases
A vast majority of respondents (92.7%)
indicate their organization’s adoption
of GenAI is in initial exploration or
Figure 2. Organization maturity level of GenAI adoption
partial implementation.
Q. How would you rate your organization’s GenAI maturity level
in terms of adoption and implementation? (n=775).

50.1%

42.6%

5.8%

1.5%

Initial exploration: Exploring but not yet implemented


Partial implementation: Some pilot projects or implementation but not widespread
Advanced implementation: Integrated into core business processes
Full implementation: Fully integrated into all aspects of the organization

7
GenAI and internal audit are inevitable partners
While survey data reflect that GenAI adoption
by internal audit is in a nascent stage, the
Figure 3. Internal audit function: GenAI activities
application of the technology to internal
audit processes is easily envisioned. Indeed, Q. GenAI in Internal Audit: Is your internal audit function involved in any of the
comments from early adopters among following activities related to GenAI? Select all that apply (n=460).
survey respondents reflect use across all
aspects of audit engagements. What’s more, 36.2%
internal audit is well-positioned to provide
supporting advisory services on AI adoption, 25.9%
governance, and controls.
12.4%
Where internal audit currently fits in
(Figure 3) 12.1%
The majority of AI work being done
6.6%
by internal auditors remains within
their functions, with 62% reporting
either using GenAI within internal audit
activities or researching future use of GenAI. Researching future use of GenAI
Using GenAI within internal audit activities
Auditing the use of GenAI within the organization
Advisory services on organization-wide
Providing advisory services for policies relating to GenAI
policies relating to GenAI and pre-
Providing pre-implementation advisory services on GenAI projects
implementation on GenAI projects is
reported by about 19% of participants.
Only about 1 in 10 respondents (12.4%)
Figure 4. Use of GenAI to support audit activities
report actual auditing of GenAI within
their organizations. Based on The IIA’s Q. You mentioned that your internal audit function is using or researching the future use
classification of AI activities, the results of GenAI to support audit activity. Please identify the ways it is being used or planned
to be used to support audit activities. Select a response for each process (Planning,
shown in Figure 3 highlight that most are
Field Work, Reporting, Follow Up). (n=410).
currently focused on viewing AI as a tool
for internal auditors (Domain 2), rather
Planning
than as a subject of an audit (Domain 1).
8% 27.6% 29.8% 12.9% 14.1% 7.6%
AI use by internal audit (Figure 4)
According to survey respondents who use Fieldwork
or are researching GenAI for future use
6% 19% 29% 18% 20.7% 7.3%
in their work, GenAI can be applied to all
phases of internal audit engagements. Reporting
The percentage who said they use/will use
GenAI “extensively” or “often” is highest for 9% 22.2% 28.3% 15.4% 18% 7.1%
planning (35.6%) and reporting (31.2%).
Follow-up
This subgroup of early adopters
6% 11.2% 17.3% 20.5% 33.7% 11.7%
provided valuable insights for those
who are looking to get started by
offering numerous use case examples
Extensively Sometimes Not at all
for all phases of a typical engagement.
Often Rarely Do not know

“Folks are thinking across the spectrum


of the internal audit process, like all the
things that we do and where can we apply
this tool,” Pelletier said. “It’s important for
internal audit, and all functions, to keep in
mind that GenAI is a tool.”

8
GenAI lacks governance
Governing policy for GenAI use (Figure 5)
More than half of respondents said there
Figure 5. Governing policy for GenAI use
is no GenAI governance policy at their
organization, or they are unsure if one exists. Q. Does your organization have a comprehensive policy governing the use of GenAI? (n=878).

Governing policy for GenAI use by


organization size (Figure 6) 12% Organization-wide
17% guidelines for all staff
With fewer than 2 in 10 (17%) reporting
clear, organization-wide guidelines for
GenAI use, the need for internal audit I ndividual department
established guidelines
advisory services is obvious. 19% but no overarching policy

“We [TeamMate] know a lot of


Not sure/not applicable
organizations haven’t established good 52%
governance practices yet,” Pelletier said. Other
“So, the opportunity is ripe for internal
audit advisory services to step in to help
the organization understand the risk.”
Figure 6. Governing policy for GenAI use by organization size

Q. Organization size: Choose the range that best describes the current number of
full-time equivalent employees (FTEs) at your current organization. Does your
organization have a comprehensive policy governing the use of GenAI? (n=878).

500 or fewer 501 to 1,500

11.1% 12.6%

18.5% 16.1%

1,501 to 5,000 5,001 to 10,000

16.6% 18.4%

18.9% 22.4%

10,001 to 50,000 More than 50,000

20.9% 36.5%

23.7% 22.2%

Clear organization-wide guidelines exist that govern use for all staff
I ndividual departments have established guidelines, but there is no
overarching policy

9
Prompt writing best practices
By all accounts, GenAI models represent an Iteration
•  Advanced Prompting Techniques
impressive technological leap. However, users Iterate and refine prompts based on model
must remember it is just another tool, and responses. Evaluate and adjust prompts to Context expansion
• 
its benefits depend largely on how well the improve the quality of outputs. To generate more complex or nuanced
tool is used, underscoring the importance outputs, consider providing additional
of human intelligence to overall success. Evaluation
•  context or specifying desired formats.
Understanding how to effectively query these Evaluate model performance by collecting
models through prompts is fundamental. user feedback and analyzing outputs to Creative prompts
• 
Below is a list of prompt writing key concepts identify areas for improvement. Don’t be afraid to experiment with writing
and best practices adapted from a video from styles, tones, or perspectives to encourage
Dr. Marc Eulerich and the Mercator Audit and Adaptation
•  more imaginative or creative outputs.
Artificial Intelligence Research Center. IIA Adapt prompts to different use cases or
members can access the full video, Prompt domains. Consider specific requirements or Zero-shot prompting
• 
Engineering for Internal Audit. constraints of the given task or application As the name implies, this technique allows
to optimize prompt strategies. models to generate responses without
Key Concepts and Best Practices being trained on specific examples; in other
Common Mistakes words, zero examples. This can be useful for
Clarity
•  outputs to novel or unexpected prompts.
Ensure that prompts are Overcomplicating prompts
• 
clear, concise, and unambiguous. Avoid using complex language or Few-shot prompting
• 
Use specific language and avoid unnecessary details. Keep prompts Similarly, this technique allows models
vague or open-ended prompts. concise and straightforward. to generate outputs with few or limited
training examples. This can be useful for
Context
•  Lack of clarity
•  outputs to new or uncommon prompts.
Provide relevant context to guide the Make sure prompts clearly communicate a
model’s understanding and generation desired outcome. Use specific instructions Chain-of-thought prompting
• 
of accurate responses. Include necessary and examples. This technique allows model outputs
information and specify desired output following a sequence of prompts. It
formats if applicable. Insufficient context
•  can be useful for outputs to complex
Provide sufficient context to help the model or multi-step prompts.
Examples
•  understand what you are looking for.
Include examples or demonstrations of the
desired output. Show both positive and Inconsistent formatting
• 
negative examples to reinforce desired Consistent formatting helps ensure
outputs. Direct the model to behave as a consistent output. Headings, bullet
type of person, process, or object. points, or numbered lists can help
improve organization.

Neglecting revision
• 
Make sure to review and revise
generated content. Refining prompts
iteratively improves the quality and
accuracy of outputs.

10
Part III:

Use cases
Input from early adopters on current • Writing
or planned usage was grouped by the GenAI can readily improve
four phases of a typical internal audit communication of issues and
engagement. Analysis of those comments recommendations by assuring that
found four categories in each phase where emails, information requests, reports,
GenAI can help: follow-ups, and other communications
are concise and accurate, relay the
• Learning proper tone, and remain consistent
GenAI can help practitioners learn about across the audit team.
potential audit subjects and issues.
A fifth use case category, Performing, is
• Brainstorming anticipated. This considers the possibility
With effective prompting, GenAI can of automating aspects of engagements
deliver impressive ideation for audit by allowing GenAI to perform part of
work in all engagement phases. the work. Because current GenAI use
by internal audit remains limited, it is
• Building not surprising that survey respondents
GenAI can help build frameworks, provided few specific examples of this.
questionnaires, and other commonly However, it is an easily envisioned
used tools. eventuality as internal audit continues
its adoption and use of the technology.

11
Use case matrices
The following use case matrices illustrate how internal audit functions are utilizing
GenAI across the four key activities: Planning, Fieldwork, Reporting, and Follow-up.

Planning phase
The planning phase of an internal audit engagement is a crucial step where the
groundwork for the audit is established. The following matrix demonstrates how
GenAI can assist with the learning, brainstorming, building, writing, and performing
aspects of this internal audit activity.

Learn Brainstorm Build Write Perform

Identify potential risks Audit objectives, scope, Create frameworks Audit objectives Automate initial
(risk assessment) and risk assessment data collection and
risk assessment

Understand Planning ideas Develop planning Work programs and Risk Automate information
new technology memoranda Control Matrices (RCMs) requests and reminders

Summarize regulations Planning frameworks Develop audit programs Policy, regulation, and
controls summaries

Research industry Generate ideas for Create risk assessment Consistent and
standards, trends, audit objectives, scope, frameworks and audit comprehensive audit
regulations, and and risk assessments planning documents plans and charters
best practices

Identify risks and controls Test procedures Build risk profiles Generate audit
from process documents and analytics programs and interview
questionnaires

General data analysis Guidance for best practices Build customized


audit programs

Identify location of Suggest key risk areas, Implement an


relevant information audit tests, usual controls, AI prompt library
and emerging risks

Analyze prior Develop Key Risk


information to define Indicators (KRIs)
new review objectives

12
Fieldwork phase

The fieldwork phase of an internal audit engagement is a key period where auditors
gather and evaluate evidence to assess the effectiveness and efficiency of an
organization’s processes. The following matrix demonstrates how GenAI can assist
with the learning, brainstorming, building, writing, and performing aspects of this
internal audit activity.

Learn Brainstorm Build Write Perform

Specialized field research Best practices Control objectives Control objectives Automate data analysis,
transaction testing, and
anomaly detection using
AI-driven tools

Data gathering Suggest data analytics Audit testing Script for audit/testing
and interrogation

Learn about specific Potential issues or areas Develop detailed audit Clear and consistent
controls, processes, of concern to focus on programs, test plans, and workpapers and
and IT systems during testing sampling methodologies documentation for
testing procedures

Sample or compare Evidence analysis Code/scripts to


audit programs for insights, tends, simplify testing of
and exceptions repeatable tasks

Contract reviews Summarize documents


for conformance by topics
to guidelines

13
Reporting phase

The reporting phase of an internal audit engagement involves compiling and presenting
the findings from the audit in a clear, structured, and actionable format. This phase is
essential for communicating the results to stakeholders and providing recommendations
for improvements. Review the following matrix to discover how peers have incorporated
GenAI into this internal audit activity.

Learn Brainstorm Build Write Perform

Study effective regulatory Scope of work Automate creation Initial drafts and reports Generate preliminary
reporting requirements of draft reports findings and
and reporting techniques recommendations
based on analyzed data

Research how others are Brainstorm Build templates for audit Review/edit drafts for Comprehensive and
reporting and providing recommendations for reports, management clarity, length structured audit reports,
support for conclusions control improvements, letters, and executive including findings,
efficiency gains, and risk summaries recommendations, and
mitigation strategies management responses

Planning ideas Standardize report formats Evaluate tone Prepare baseline


on issues drafts draft reports

Issue criteria and Experiment with Strengthen findings Create audit report
development AI video reporting with industry facts summaries based on
detailed findings

Create vivid presentations Summarize and find


alternative phrasing

Provide inputs and


trend analysis of issues
and control gaps

14
Follow-up phase

The follow-up is crucial for verifying that corrective actions have been taken and
improvements have been made. The following matrix demonstrates how GenAI can assist
with the learning, brainstorming, building, writing, and performing aspects of this internal
audit activity.

Learn Brainstorm Build Write Perform

Learn about Brainstorm strategies Create follow-up Follow-up reports and Automate the tracking
follow-up procedures for ensuring frameworks, tracking status updates, clearly of remediation
and best practices management implements mechanisms, and documenting progress efforts and follow-up
recommendations dashboards on agreed on remediation efforts testing to confirm the
action items implementation of
recommendations

Research IIA Standards Combine management Craft emails and send Schedule follow-ups
and best practices responses in various notes to business lines and monitor audit
audit weaknesses to for follow-up recommendations
provide patterns and
thematic insights

Escalate overdue Generate reports Monitor progress of action


recommendations on the status of plan status and validate
not yet implemented unresolved issues issue closure evidence

Track undisclosed issues Follow-up responses

Client response assessment

Application of
validation testing

Track follow-up dates


and remediation

Send automated reminders

15
Part IV:

Next steps
Caution is not an excuse for inaction
Concerns about GenAI bias and fraud risks “It’s not like we’re looking at AI as a
were articulated in comments from several replacement of humans,” he said. “We’re
survey respondents. These are legitimate looking at it as an augmentation to help
concerns (see sidebar, AI Limitations and humans be better, to be more efficient, to
Pitfalls). The risks associated with GenAI be more productive, to be better at things
use include cybersecurity; data protection; that you do day-to-day. The collaboration
lack of transparency; and unknown bias, or interaction between human intelligence
discrimination, and ethical questions about and AI lead to the best results. “
AI algorithms. All provide ample reasons to
pause. As with any new technology, costs, Becoming comfortable with the technology
data quality, and organizational resistance also positions internal audit to provide
to change pose additional challenges to its relevant and timely advice as organizations
use. What’s more, internal audit’s general develop strategies to integrate AI into
resistance to being early adopters of new business operations and processes.
technology poses a barrier.
“Auditors are naturally curious; leverage
However, waiting is not an option, that to become knowledgeable enough so
particularly at a time when the dynamic that you are part of the conversation, and
risk landscape is evolving at an ever- not sitting on the sidelines,” Pelletier said.
increasing pace. “Making the perfect the
enemy of the good,” as one respondent One of the principal challenges faced by
commented, is not an option. practitioners is overcoming an ingrained
resistance to risk-taking. Often, leaders in
GenAI is a next-level tool that can create internal audit who are averse to risk prefer
efficiency and boost productivity within to wait and observe the evolution of new
internal audit and across the organization. trends. Unfortunately, this can lead to
“It’s time to be bold,” Pelletier urged. internal audit functions consistently having
“Internal audit leaders need to embrace to play catch-up. What’s more, this approach
this opportunity, learn about AI, and could contribute to other areas of the
adopt its use in their audit processes organization pushing ahead without proper
for the benefit of their function and the controls in place or otherwise accounting
organizations they support.” for AI pitfalls. Ironically, in such instances,
internal audit could be contributing to AI
Beyond its ability to enhance efficiency and risks instead of helping to minimize them.
productivity, GenAI holds great promise for
improving internal audit quality, Pelletier The recommendation is to avoid waiting and
said. For example, large teams that struggle instead be proactive. For instance, reassess
with alignment on writing can leverage hiring strategies to incorporate diverse skill
AI to make reports consistent in their sets, enhance the capabilities of the existing
language, brevity, and tone. team through training, and engage external
partners to address any gaps.

16
Gen-AI limitations and pitfalls
The opportunities AI presents are abundant Hallucinations Data Protection
and clear, but they come with attendant The strength of GenAI models is that they Another strength of GenAI models that is
ills, including concerns about cybersecurity, can tap into vast amounts of online data to also a pitfall is its ability to incorporate all
data privacy, ethics, its potential to displace analyze, synthesize, and generate impressive the data it is fed into its store of resources.
workers, and more. Additionally, the current outputs. That also has proven to be a pitfall. Well-intentioned workers can unwittingly put
technology carries with it inconsistencies and There are abundant examples of GenAI out sensitive or proprietary information into
glitches that if not recognized and considered responses incorporating false or incorrect the online ether by using publicly available
can create significant risks. information from questionable sources. This GenAI tools. This makes strong governance
could lead to significant problems should policies critical for any organization looking
The following provides a partial list of AI faulty AI outputs be used in publicly facing to use GenAI.
downsides and pitfalls. content or for strategic decision making.
Corrective actions
• 
The Black Box Dilemma Corrective actions
•  Develop and disseminate AI governance
AI models are built on complex algorithms Validate findings, especially for critical policies and controls (wherever possible).
that can be difficult to understand, even for initiatives, with known SMEs. Leverage both Consider developing and implementing in-
those who are experts on the technology. This internal and credible external resources. house GenAI tools with restricted access.
lack of transparency is often referred to as Following prompting best practices also will
the “Black Box” aspect of AI. Not knowing how help reduce erroneous results. Cybersecurity
algorithms are designed opens a Pandora’s The wonders of GenAI to improve efficiency
box of questions about bias, ethics, and and productivity also make it a valuable tool
potential unknowns that could have negative for hackers looking for innovative ways to
impacts on AI’s outputs. For example, breach cyber protections and steal or hold
one study found that error rates of facial hostage valuable data.
recognition technology were significantly
higher for darker-skinned women than for
light-skinned men.

Corrective actions
• 
Ensure executive management and the
board are aware of this shortcoming by
making sure internal audit’s voice is heard
and represented on any organizational
AI task force. If one does not exist,
recommend its creation.

17
Conclusion
The introduction of user-friendly The future of internal auditing
GenAI models stunned the business will inevitably involve a mix of
world out of its post-pandemic AI tools, technological aids, and
torpor. Indeed, its ease of use, human expertise. AI offers unique
practical applicability across a applications and a promising
wide range of business processes, outlook for modern internal audit
and near zero cost for functions. However, the one factor
implementation was the perfect that remains unclear is whether
antidote for the nearly two-year internal audit can overcome
slowdown in operations. What’s its past reticence to embrace
more, it quickly became evident new technology and effectively
that beyond its energizing impacts, incorporate AI into its processes.
the tool’s ability to automate
repetitive, mundane work and help The findings from the TeamMate and
manage ever-growing expanses of Foundation global survey provide a
data had genuine potential to valuable snapshot into the current
transform how business is done. state of adoption and use of GenAI.
What it reveals is a business world
The cutting-edge technology will on the precipice of a technological
irrevocably change any profession paradigm shift. Internal audit
that relies on data and data analysis, functions that can quickly, safely,
including internal auditing. While and effectively integrate AI into their
the ultimate solution to the AI riddle digital processes will hold a huge
remains somewhat clouded, clues advantage over those who delay.
are emerging as to how it can be The time to act is now.
applied to improve internal audit
efficiency, productivity, and quality.

18
Appendix: survey demographics
The charts below provide information on the characteristics of the respondents who
participated in a survey.

Role Organization type Organization size

Q. Which of the following best describes your Q. What type of organization do you currently Q. Choose the range that best describes the
role in your current organization’s internal work for? (If you are a service provider, please current number of full-time equivalent
audit function? (n=924). answer based on your primary client.) (n=889). employees (FTEs) at your current
organization. (n=924).

29% 29% 29% 30%

23% 23% 24%


22%
18%
16%

11% 11%
4% 8% 4%
2% 2% 1% 7% 1%
5%

 hief audit executive or head of internal audit


C Nonprofit  00 or fewer
5
Director or equivalent Privately held 501 to 1,500
Senior manager/manager Public sector (government) 1,501 to 5,000
Staff Publicly traded 5,001 to 10,000
Service provider Service provider/consultant 10,001 to 50,000
Other Not sure/not applicable More than 50,000
Not an internal auditor Other Not sure/not applicable

Region Internal audit function size

Q. In what region is your current organization Q. How many full-time equivalents (FTEs) are
based? (n=924). in your internal audit function? (If you are
a service provider, please answer based on
your primary client.) (n=924).

34%

26%
23% 23%
20% 19%
16%
13%

8% 9%
7% 2%

Africa  to 3
1
Asia Pacific 4 to 9
Europe 10 to 24
Latin America and the Caribbean 25 to 49
Middle East 50 to 50+
North America Not sure/not applicable

19
About The Institute of Internal Auditors
and the Internal Audit Foundation
The Institute of Internal Auditors (The IIA) is an international professional association that
serves more than 245,000 global members and has awarded more than 200,000 Certified
Internal Auditor® (CIA®) certifications worldwide. Established in 1941, The IIA is recognized
throughout the world as the internal audit profession’s leader in standards, certifications,
education, research, and technical guidance. For more information, visit theiia.org.

The Internal Audit Foundation is an essential global resource for advancing the internal
audit profession. Foundation-funded research provides internal audit practitioners
and their stakeholders with insight on emerging topics and promotes and advances the
value of the internal audit profession globally. In addition, through its Academic Fund,
the Foundation supports the profession’s future by providing grants to students and
educators who participate in The Institute of Internal Auditors’ Internal Auditing
Education Partnership program. For more information, visit theiia.org/Foundation.

Copyright © 2024 by the Internal Audit Foundation. All rights reserved.

2024–25 Board 2024–25 Committee


of Trustees of Research and
President Education Advisors
Warren W. Stippich, Jr., CIA, CRMA
Chair
Officers Nora Kelani, CIA, CRMA
Glenn Ho, CIA, CRMA
Nora Kelani, CIA, CRMA Members
Shirley Livhuwani Machaba, CCSA, CRMA Tonya Arnold-Tornquist, CIA, CRMA
Christopher Calvin, PhD, CIA
Trustees Joseph Ian Canlas, CIA, CRMA
Jose Gabriel Calderon, CIA, CRMA Andre Domingos
Hossam El Shaffei, CCSA, CRMA Christina Duquette, CRMA
Susan Haseley, CIA Marc Eulerich, PhD, CIA
Dawn Jones, CIA, CRMA Dagmar Flores, CIA, CCSA, CRMA
Michael A. Smith Anargul Kairulla, CIA
Subramanian Bhaskar Ayaka Mitsunari
Reyes Fuentes Ortea, CIA, CCSA, CRMA Ahmed Shawky Mohammed, DBA, CIA
Anthony J. Pugliese, CIA Grace Mubako, PhD, CIA
Ruth Doreen Mutebe, CIA
Staff Liaison Thomas O’Reilly
Laura LeBlanc, Emmanuel Pascal, CIA, CRMA
Senior Director, Internal Audit Foundation Brian Tremblay, CIA
Koji Watanabe
Stacy Wright, CIA

Staff Liaison
Nicole Narkiewicz, Ph.D.
Director, Academic & Research Strategy

20
About Wolters Kluwer
TeamMate
As part of the CP & ESG division of Wolters Digital
Kluwer, TeamMate delivers award-winning Harness the latest technologies to
internal audit and analytics solutions to demonstrate confidence in your coverage,
corporate internal audit departments and improve productivity, and boost agility to
public sector audit organizations around address business changes.
the world. As internal audit teams evolve
to deliver deeper insights, greater risk Integrated
assurance, and improve efficiency, they Connect critical business systems and
require purpose-built and future-ready key stakeholders to eliminate siloed
solutions. TeamMate provides expert decision-making and improve
solutions internal auditors rely on to drive organizational performance.
value into their organizations.
Analytical
End reliance on incomplete, fragmented
evidence. Implement integrated analytics
to uncover hidden risks and obtain
objective, comprehensive results.

21
Contact information
Americas Please visit tm.wolterskluwer.com
4221 W Boy Scout Blvd #500 for more information.
Tampa, FL 33607
U.S.A.

Europe, Middle East, and Africa


8th Floor
30 Churchill Place
Canary Wharf
London
E14 5RE
United Kingdom
Phone: +44 20 3197 6566

Asia Pacific
5 Shenton Way
#20-01/03 UIC Building
Singapore 068808

Copyright © 2024 Wolters Kluwer Financial Services, Inc.

You might also like

pFad - Phonifier reborn

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


Alternative Proxies:

Alternative Proxy

pFad Proxy

pFad v3 Proxy

pFad v4 Proxy