All Things Internal Audit AI Podcast: Generative AI Uses for Internal Audit

Summary

In this episode of the All Things Internal Audit AI podcast, Sue King from KPMG and Grant Osler from Worka explore the impact of generative AI on internal audit processes. They discuss best practices for using AI, including specificity in prompts and leveraging AI for risk assessment, report generation, and root cause analysis. The conversation highlights AI's potential to enhance audit efficiency, shift from assurance to advisory roles, and contribute to more impactful and value-added auditing practices.

Takeaways

• 📝 Generative AI can significantly speed up internal audit processes and enhance risk management in evolving areas.
• 🔍 The more specific and contextual the prompts given to AI tools, the better the results they produce.
• 💡 Practicing and experimenting with different prompts can help fine-tune the AI's responses and improve its accuracy.
• 📚 AI can be used to assist in planning audits, identifying risks, and suggesting controls and procedures.
• ✍️ AI excels in drafting reports, allowing auditors to focus on higher-value tasks rather than spending time on report writing.
• 🔑 With minimal data entry, AI can generate detailed recommendations and action-oriented insights, enhancing audit value.
• 🔍 AI can ingest various document formats for analysis, enabling auditors to perform testing procedures more efficiently.
• 🛠️ Auditors can use AI for root cause analysis and to provide actionable recommendations to control owners.
• 🚀 AI can help auditors transition from assurance to advisory roles, amplifying their impact on improving operations and reducing risk.
• 🌐 AI platforms can be connected to enterprise systems via APIs, allowing for real-time monitoring and data analysis.
• 🔮 The future of internal audit with AI is expected to focus on value-added services, root cause analysis, and improved risk management.

Q & A

What is the main topic of the podcast episode?

The main topic of the podcast episode is the potential of generative AI to speed up internal audit processes and support risk management in evolving risk areas.

What is the importance of specificity when entering prompts into generative AI tools?

The more specific you can be when entering prompts, the better the results you will get from the AI. It allows for more accurate and contextually relevant outputs.

How can generative AI assist in the auditing process?

Generative AI can assist in the auditing process by speeding up report writing, helping with risk analysis, and providing recommendations for control improvements.

What is one way generative AI can help with data entry in auditing?

Generative AI can take in documents in various formats like Excel or PDF, and perform tasks such as comparing and contrasting data, which reduces the amount of manual data entry required.

How does generative AI support risk management in internal audits?

Generative AI supports risk management by identifying potential risks in specific areas, suggesting important controls, and helping auditors understand how these risks might manifest.

What is the role of generative AI in report writing for internal audits?

Generative AI can write initial drafts of reports, which auditors can then refine. This helps to save time and allows auditors to focus on more value-added tasks.

Can generative AI generate work programs for auditors?

Yes, generative AI can generate work programs, helping auditors to plan and structure their testing procedures more efficiently.

How can generative AI be used for root cause analysis in auditing?

Generative AI can analyze data and provide insights into the root causes of issues, allowing auditors to offer more targeted and actionable recommendations.

What are some creative uses of AI by auditors mentioned in the podcast?

Some creative uses include using AI to document walkthroughs, compare process narratives to actual processes, and quickly identify differences and areas for improvement.

How does generative AI impact the role of auditors in terms of value addition?

Generative AI allows auditors to focus more on value-added activities such as advisory services, control improvement suggestions, and strategic planning, rather than getting bogged down in data entry and report drafting.

What is the potential future of internal audit's use of AI according to the podcast?

The future of internal audit's use of AI could involve more automation, continuous monitoring, and a greater focus on value-added services, with AI serving as a tool to enhance auditors' impact and efficiency.

How can generative AI help with the governance and control of AI usage in other departments?

Generative AI can help auditors understand the governance and control mechanisms over AI usage in other departments, ensuring that there are proper guardrails in place and that the usage is aligned with company policies.

What are some challenges auditors face when writing business cases and recommendations?

One of the challenges auditors face is making the business case and recommendations compelling and actionable. Generative AI can assist in crafting these in a more effective and persuasive tone.

Outlines

00:00

🤖 Leveraging AI for Internal Audit Efficiency

In this podcast episode, Sue King from KPMG and Grant Osler from Worka explore the potential of generative AI to expedite internal audit processes and enhance risk management. They discuss best practices for crafting prompts to get the most accurate results from AI tools, emphasizing the importance of specificity and context. The conversation highlights the use of AI in planning, report writing, and drafting recommendations, as well as its ability to analyze large volumes of data and perform testing procedures. The episode underscores the shift from assurance to advisory roles for auditors, with AI facilitating more value-added contributions to the first line of defense.

05:00

🚀 Creative AI Applications in Auditing and the Future

The second paragraph delves into creative uses of AI by auditors, such as documenting walkthroughs and comparing process narratives to identify discrepancies quickly. It discusses the potential of AI to alleviate mundane tasks, allowing auditors to focus on higher-value activities. The speakers also touch on the next frontier for internal audits, including moving upstream in the audit process and providing more impactful insights. They predict that AI will continue to evolve, enabling auditors to offer more detailed and insightful recommendations, while also emphasizing the importance of professional oversight to ensure accuracy.

10:01

🔮 Envisioning the Integration of AI in Internal Audits

The final paragraph speculates on the future of AI in internal audits, with a focus on the rapid pace of technological advancement and its implications for the field. It suggests that AI will not replace auditors but will instead augment their roles, making them more valuable by handling repetitive tasks and enabling deeper analysis. The discussion also addresses the importance of auditors understanding AI to advise on its risks and governance within organizations. The speakers advocate for auditors to familiarize themselves with AI to have informed conversations about its application and control within the business, highlighting the transformative impact of AI on the auditing profession.

Keywords

💡Generative AI

Generative AI refers to artificial intelligence systems that can create new content, such as text, images, or audio, that is similar to, but not identical to, existing content. In the video's context, it is discussed as a tool to speed up internal audit processes and support risk management. An example from the script is when the speakers discuss how generative AI can be used to write better and more efficiently than humans, particularly in drafting audit reports.

💡Internal Audit

Internal audit is the process by which an organization evaluates its own controls and risk management processes to ensure they are effective and efficient. The video focuses on how generative AI can enhance this process. For instance, the script mentions using AI for planning, recommendation, and testing procedures within an audit.

💡Risk Management

Risk management involves the identification, evaluation, and prioritization of risks followed by coordinated efforts to minimize, monitor, and control the probability or impact of unfortunate events. The video discusses the potential of generative AI in evolving risk areas, such as identifying risks in new areas and suggesting controls and procedures to mitigate them.

💡Auditor

An auditor is a professional who performs audits to provide an independent examination of an organization's financial statements or other operational areas. The script discusses the impact of AI on auditors' work, such as reducing the time spent on data entry and report writing, allowing them to focus more on value-added tasks.

💡Control Owner

A control owner is an individual responsible for the implementation and maintenance of controls within an organization. The video mentions that AI can help auditors communicate more effectively with control owners by providing a deeper analysis and actionable recommendations, such as root cause analysis of issues.

💡Data Ingestion

Data ingestion is the process of importing data into a system. In the context of the video, AI systems can ingest various document formats, such as Excel or PDFs, to perform analysis and testing procedures, which helps in identifying issues like terminated users with active access.

💡APIs

APIs, or Application Programming Interfaces, are sets of routines, protocols, and tools for building software applications. The script mentions that systems designed with APIs can connect to other systems and bring in data for AI to analyze, which is crucial for integrating AI into existing enterprise systems.

💡Continuous Monitoring

Continuous monitoring is a process that involves ongoing tracking and analysis of systems and activities to ensure they are running as expected. The video suggests that while AI may not be the perfect solution for continuous monitoring, it can be used to prove the value of certain analyses, which can then inform the choice of technology for ongoing monitoring.

💡Root Cause Analysis

Root cause analysis is a method of problem-solving used to identify the underlying causes of faults or issues. The video emphasizes AI's ability to assist in root cause analysis, providing auditors with insights that can lead to more effective recommendations and control improvements.

💡Regulation

Regulation refers to the rules or directives made and maintained by an authority. The script discusses the rapid pace of regulatory changes, such as the Cyber rule and the European AI rules, and how AI can help auditors keep up by summarizing and highlighting key points from policies and regulations.

💡Policy

A policy is a document or set of guidelines that outlines how to achieve an organization's goals and objectives. The video mentions AI's ability to read and summarize policies, helping to identify mandatory actions and create checklists for compliance, which is a valuable tool for auditors.

Highlights

The potential of generative AI to speed up internal audit processes and support risk management in evolving risk areas is discussed.

Best practices for entering prompts into AI tools include being as specific as possible and providing context for better results.

The importance of rewriting prompts and comparing results to fine-tune AI responses is emphasized.

AI can be used to identify and deal with 'hallucinations' or inaccuracies in its responses by asking for references.

Common uses of AI by internal auditors include risk assessment, control identification, and procedure development.

AI can significantly reduce the time spent on report writing by providing well-crafted drafts for auditors to refine.

The minimal data entry required for AI to generate draft reports is highlighted, showcasing its efficiency.

AI's ability to generate work programs and perform document analysis for testing procedures is discussed.

The shift from assurance to advisory roles for auditors, enabled by AI, is noted as a significant development.

Creative uses of AI by auditors, such as documenting walkthroughs and comparing process narratives, are explored.

The impact of AI on auditor retention and the ability to move into higher value roles within organizations is considered.

The next frontier for internal audits using AI includes moving upstream in the process and providing more value-added insights.

AI's role in helping auditors write better business cases and recommendations is highlighted.

The importance of auditors being familiar with AI to understand and advise on its use within the company is underscored.

Connectivity of AI platforms to enterprise systems for monitoring and analyzing data through APIs is mentioned.

The vision for internal audit's use of AI in five years, focusing on technology solutions and continuous improvement, is discussed.

The necessity for auditors to understand AI risks and governance to advise the company effectively is emphasized.

The transformative effect of AI on the speed and efficiency of internal audits, making the job more enjoyable, is noted.

Transcripts

[Music]

The Institute of internal Auditors

presents all things internal audit AI

podcast in this episode Sue king of KPMG

and Grant Osler of worka discuss the

potential of generative AI to speed up

internal audit processes and support

risk management in evolving risk areas

hi I'm here with Grant from worka and

Sue from KPMG and we're talking about

generative AI what are some best

practices for uh entering prompts into

the uh into the tool oh that's a great

question so I think you know just being

as specific as you can be right I mean

you can you can write a very very simple

or open plain language prompt but the

more specific you can be of like compare

it to this framework or like specific to

like you were saying like specific to

audit or like uh yeah the more

specificity the better announc you're

going to get the more context you

provide the better you're going to get

results back and one we were talking

about before was things in preparing for

a session we talk this I like to like

I'll rewrite the prompt three or four

different ways I'll tweak it and I'll

compare the results back oh okay these

things are common okay I know that's

important these things are outliers

where am I going and ask for references

like where did you get this from so you

can take some of the hallucinations it's

going to do it but I can identify him

and I can deal with them a lot easier

will it cite its references it will if

you ask it right and it's and you don't

have to you don't have to be that

articulate in doing it it fills in the

blanks pretty well but the more you do

it just like anything else the better

you get so practice a lot yeah and the

more you can be you know tell it to like

okay that's great summarize it or what

are the highlights out of this like to

to really get it to fine tune fine tune

announces you don't end up with pages

and pages right and what what are some

of the most common uses of artificial

intelligence by internal Auditors what

what are the what are you seeing out

there how are people we're seeing a lot

n a lot of lot of work on the front and

say look what are the risks in this area

what are the most important ones how are

they going to be how would what controls

would I look for in that like what kind

of procedures would I do for that so A

Lot in the planning is really big we see

a lot on the back end I don't know about

anybody else but we used to spend a lot

of time word smithing reports gen AI

writes better than we do most of the

time let it take the P then you go in

and refine it it's hard to start with a

blank piece of paper right but if I can

start with something that's pretty well

crafted I can then go in and adjust it

and bring it to where I need to be

really quickly it frees your time up to

do more value added audit instead of

spending your time word smithing reports

and how much data entry does it take to

to get it or you know to get it to uh

write that draft report like how much uh

prompting do you have to give it I yeah

I don't feel like it really needs that

much is like you know I've got this

exception write me a recommendation uh

you know make it action oriented make it

detailed right you can put all there do

you want it in bullet you can tell it

what you want and it will give you what

you ask for it's amazing so I think

that's that's one of the great use cases

right is like the the planning and you

know making you smarter about you going

to a new area that you don't really know

as much about that's kind of more

technical or you know specialized to an

industry like make helps you get smarter

in that planning phase um and then as

Grant says like gets you to that

recommendation does it have the ability

yet to generate the work programs

absolutely it does absolutely it does so

so I think yeah so doing the planning

the recommendation but then I think you

know actually doing the testing as well

to you can you can give it documents to

ingest um you know be they you know

whatever format Excel PDF or whatever

and you can tell it like you know

compare and contrast so you can start

doing some testing procedures to say you

know like okay like the example that we

went through in our session yesterday is

around terminated user testing right do

I have people who are who've got AC

active access that are not employees but

then the beauty of AI is that then you

can say like okay well once I found

people that should have been terminated

then you know you can give it another

data source and say well go and look and

see did that person actually do anything

after their termination date or like you

can give it the tickets and say was the

ticket issued for that terminated user

and it just didn't get executed on so

you can start getting into a lot more of

those um kind of what if and look back

type analysis very easily there been

been a focus for a long time of like how

do we get to the point where internal

Auditors are more Val value added right

and so I think really being able to go

to a control owner not only say like oh

you've got an exception which okay they

hate hearing that but then if you've

been able to get to the next level and

say well I've done root cause analysis

the issue is like the tickets are

getting issued but they're not getting

closed out properly right or um or even

saying like hey I did this analysis if

you first line were to were to take that

activity this would be your control

activity and here's how we would how we

would resolve it so I think getting back

getting back to that you know action

oriented value added to the the first

line I think that's a huge point being

able to shift from Assurance to advisory

and help the first and second line do

their job better we're able to amplify

the impact we have it's huge and so the

more we can help this move Upstream the

more impactful we are as Auditors what

what are some of the most creative uses

that you've seen of of AI by Auditors

you know how uh have you seen anything

where you know you're surprised that uh

what it could I think there's I think

there's a lot of different things right

so we're you know we're experimenting

with using using co-pilot for example to

uh to document uh from a walkthr that

we're having and then you know it can

create your first draft of a walkthrough

and then you can say like okay now

compare the walkthrough of what the

process owner just described the the

process to me compare that to the

process narrative or to the rackam or to

the flowchart and tell me why there are

differences

and and you know the the beauty of it is

the speed that then you can go back to

the control owner you know the next day

and say hey when we talked about this

you never you didn't touch on this like

was that we just omitted it or has it

changed right as opposed to you know

historically it might have taken you

know a week or to get back to it and

then the the control owner is like I

don't remember what we talked about so

so you know some of it is uh as Grant

said you know kind of takes takes some

of the uh kind of more the grunt work

kind of out of this right so that we are

more focused on on valette um but the

speed thing is is also terrific I think

that point about taking some of that

that busy monotonous workout every

auditor I know every audit leader is

trying to staff their team with the

right people and it's a struggle right

now if you can get your people working

on the right things a they like their

job more nobody likes writing narratives

I don't maybe they is but I don't know

they never work for me right so take the

stuff they don't like to do automate as

much of that you can let them focus on

things that adds more value that is more

fun for them our retention will get

better our ability to keep them and move

those people into higher value roles in

the organization gets better so it's

it's kind of that virtuous cycle is my

opinion and what do you see as like the

next Frontier for uh internal audits use

of AI right where where are we going

next well I just you know I think going

back to what I was saying earlier right

I feel like we've all been stuck you

know doing socks or or doing some of the

kind of like the standard AUD

um you know but I think this tool really

enables us to to speed up and again like

really get into that value added and and

to be able to use that knowledge to say

like okay something that maybe I might

have needed a specialist to help me like

you know you've got much more of a of a

jump start but I do think that whole

like value added piece um and and more

insight being able to to really dig into

that uh that root root cause analysis

that we've all been talking about for a

long time I think one of the challenges

Auditors have is

writing a real business case in a

recommendation right really make sale

and and we don't always do that very

well you can ask J to write it in a tone

where it will do a much better job

that'll drive adoption if if things

don't change we Rite up a finding we're

of no value let's just be honest we're

of value when the operation gets better

and they change and they have less risk

because they've got better controls or

other things in place and I think J can

help in that a lot and it's going to

keep getting better can n AI make

recommendations at this point and you

know like Beyond just the high level

generic can they you know can can it

really generate some insightful

recommendations yeah yeah we've seen it

we've seen it do that yeah at a detailed

control or control level right yeah for

sure but it's not on autopilot right you

still want the professional who has

experience to say does this make sense

is that right based on what we saw right

because it Charles King who is from KPMG

who was on our group yesterday was ready

was like it's a huge algorithm right

it's it's doing math you need to make

sure that that it's still on point right

so I think that again I don't see this

replacing Auditors I see this letting

Auditors be a whole lot more important a

lot more impactful right yeah yeah I I I

totally agree right it's like that first

version but we all know it's way easier

to edit something than it is to be

staring at a blank sheet of paper are

scary right sure uh are there ways to

connect uh an AI platform to your

Enterprise systems so that the AI can be

monitoring or analyzing the data yeah so

and I can speak to workas which I can't

speak any but you know our system is

designed with apis to connect to other

systems and bring the data in and and

because the Gen is Right native in our

application I can go right to that data

and I think it's there so yeah it's

really there

um understanding what data you need and

getting the right Connections in if I

don't have a connection and maybe it's

not something I'm going to do

repetitively I can bring it in in the

form of a spreadsheet or something like

that so it's amazingly simple actually

to bring the data in and connect up to

it which is really fun yeah so

ultimately how do you see uh internal

audit incorporating artificial

intelligence into its work you know five

years from now W how would what would

you envision you know internal audits

use of AI to be well it's interesting we

were talking about right you know like

things are moving so quickly so five

years by five years from now like gosh

we won't be on the moon but um you know

but I I do think like this this you know

there's been this push like how do we

how do we use more technology and more

Automation in the way that we're aiting

whether it's internal AIT or whether

it's socks and I think this has really

given us that you know that low code no

code solution to be able to start doing

analysis you know I think you know like

is it the is it the perfect solution if

you want to do continuous monitoring or

whatever maybe not but what it allows

you to do is to do that you know like

okay well I I did it using j i and I

proved that like this is going to be

valuable and useful so now let's go and

figure out like what is the right

technology solution so I I just think um

yeah it it's got a myriad of

possibilities um it's not going to as

Grant says not going to replace Auditors

but really allows us again to be much

more valuable for the first line uh

which I think is terrific yeah I mean

it's it's what 15 or 16 months old now

and we're in a whole different world so

I don't I'm not that good at

prognosticator say 5 years I'm not I

know what a year from now looks like but

but what I see is that as people get in

and use it they're

learning AI like everything else comes

with risks and if you're not in it using

it your audit team you're not

understanding those risks well enough to

really advise your company and you're

not helping your company I can guarantee

you somebody in marketing or somewhere

else is using it and if you're not

helping put the right governance in

place around this you're not doing your

job as an otter in my opinion right so

you've got to be well enough versed to

have that conversation with the other

people to put things in place have those

right guard rails in place I think

that's maybe the next step for us I

don't know how many three or four steps

I don't know what that looks like I

think right now it's get in get familiar

use it practice and use it so that you

can really sit down and have an

intelligent conversation with your

business say look these are risks here's

how we're addressing them what are you

doing how do we do this and really put

things in place by the way I'll help you

write the policy and just uh one thing

that that Sparks so the other thing we

haven't talked about right we're talking

about using gen

but I think as Auditors exactly as Grant

says you have to be familiar with it

because you know the rest of the

business is going to be using it and so

we need to look at it we need to come at

it from a controls perspective as well

right and say like okay what's the

governance we've got over this let's

make sure that we're managing it are

there any other uh you know things that

you know you're most excited about in in

Ai and like you know bringing it to your

clients or incorporating it into your

platform what what what's the uh you

know the next Target nobody knows better

about chasing down data make sure it's

complete and accurate than Auditors

that's what we do every day and so I

think there's Frontiers in partnering

with the business and all these things

in these new emerging risk areas right

where we don't know but it can help us

get speed so we can again have those

really meaningful conversations with

people who do know it and help them get

there faster this is about us helping

the first and second line be more

impactful that's our job you know I

think when you look at the speed that

regulation is coming at us right so

whether it's you know the Cyber rule

which we've still got to deal with right

the new climate rule we've got uh the

European AI rules I mean it's just like

coming at us but um you know being able

to use gen to to summarize it get me to

the high points some somebody in our

session uh the other day was talking

about policies of like you can tell it

read a policy and find all of the all of

the must must do right what like so then

you can come up with a list so I can use

a checklist so like again just that

ability of it to consume information and

summarize it for you is just is just

awesome it's lifechanging I mean it

really is if I had known this I might

still be auditing instead of doing what

I do now just it's it's making it fun

well thank you grant and sue for talking

to us about artificial intelligence and

uh it's been very informative terrific

thanks for having us apprciate

thanks if you like this podcast Please

Subscribe and rate US you can subscribe

wherever you get your

podcasts you can also catch each episode

on YouTube or at the.org that's th

H a.org