All Things Internal Audit: Risk & Cyber Audit Opportunities with AI

Summary

In this episode of 'All Things Internal Audit Tech', thought leaders discuss the transformative role of AI in risk and cybersecurity audits. David Petrisky and Brian Willis from the Institute of Internal Auditors and LBMC highlight how AI models, trained on specific compliance documents like PCI, offer informed and accurate responses. Canal Agrawal from Diligent underscores AI's utility in continuous risk assessment, scenario analysis, and enhancing communication across departments. West Blocki and Ethan Rohani from Grant Thorton emphasize AI's efficiency in dynamic risk assessment, while Brian Willis showcases generative AI's potential in cybersecurity, offering audit accuracy, consistency, and cost reduction.

Takeaways

• 🧠 AI is being trained on specific organizational documents to understand controls and operations for compliance programs, such as the development of a PCI GPT at LBMC.
• 🗣️ AI's text analysis capabilities are enhancing the process of interviews and surveys, allowing for pattern recognition and more informed risk assessments.
• 🔍 AI facilitates continuous risk assessment, identifying the highest risks in real-time rather than waiting for periodic assessments.
• 📊 Scenario analysis is being improved with AI by combining data from various departments to simulate different risk scenarios.
• 📝 AI is aiding in risk scoring, communication, and engagement within and outside the audit department.
• 🛠️ A dynamic framework model is being developed to score and rank risks, allowing for customized input based on an organization's unique framework.
• 🌐 AI enables 24/7 risk assessment conversations, improving efficiency and employee satisfaction by removing time zone barriers.
• 🔒 Generative AI, like chat GPT, is a promising tool for cybersecurity audits, providing a knowledgeable base similar to an experienced team member.
• 📚 AI tools offer audit accuracy and consistency by providing direct access to documented information without the need for manual search.
• 💰 The cost of compliance is reduced through AI, as it expedites the review of documentation and evidence, saving man hours in the audit process.

Q & A

How is AI being integrated into compliance programs according to the transcript?

AI is being used to train models on specific organizational systems and controls, such as creating a custom GPT trained on PCI documentation, to understand and provide informed answers about compliance requirements.

What is the significance of training an AI model like GPT on specific information?

Training an AI model on specific information allows it to provide well-informed, accurate answers based on the trained documents, reducing the likelihood of providing incorrect or fabricated information.

How can AI assist in continuous risk assessment as mentioned by Canal Agrawal?

AI can analyze text data from interviews and surveys to identify patterns, support continuous risk assessment by identifying the highest risks in real-time, and aid in scenario analysis by integrating data from different departments.

What are some of the practical applications of AI in enhancing risk assessment processes?

AI can be used for risk scoring, improving communication about risks across departments, and developing dynamic frameworks for weighted scoring based on an organization's specific industry and business needs.

How does AI support audit accuracy and consistency in the context of cyber security?

AI tools like chat GPT provide access to a reliable knowledge base of documented information about cyber security audit and compliance, ensuring that the information obtained is accurate and consistent.

What benefits does AI offer in terms of cost of compliance in audits?

AI can reduce the time spent on audits by quickly searching through documentation and providing instant answers, thereby decreasing the cost of compliance in terms of both time and expenditure.

How can AI improve communication within and outside the audit department?

AI can help in ensuring that both audit teams and other departments are aware of risks, facilitating better communication and understanding of potential issues.

What is the potential impact of AI on employee satisfaction in a global company?

AI can enable employees to have conversations and preliminary discussions at any time, streamlining communication and avoiding the need for inconvenient scheduling, thus increasing employee satisfaction.

What is the role of generative AI in cyber security audit and compliance?

Generative AI serves as a knowledgeable team member that has access to all documented information about cyber security audit and compliance, providing a conversational interface for accessing this information.

How does AI contribute to a more efficient and dynamic risk assessment process?

AI can perform risk evaluation, identification, impact assessments, and scoring analysis in a dynamic framework model, allowing for a more efficient and tailored risk assessment process.

What is the potential of AI in facilitating deeper and more useful conversations in audits?

AI can be used to conduct preliminary discussions, gathering information that can streamline deeper and more useful conversations when humans interact, improving the overall audit process.

Outlines

00:00

🤖 AI in Compliance Programs and Cybersecurity Audits

This paragraph discusses the integration of AI into compliance programs and cybersecurity audits. David Petrisky and Brian Willis explore the use of AI models like chat GPT, which can be customized and trained on specific organizational information. They highlight the development of a PCI GPT at lbmc, trained on PCI documentation to provide informed answers on compliance requirements. The paragraph also touches on the broader applications of AI in risk assessment, emphasizing the efficiency and accuracy AI brings to the auditing process.

05:00

🔍 Enhancing Risk Assessment with AI

The second paragraph delves into how internal auditors are leveraging AI for risk assessments. Canal Agrawal explains the utility of AI in analyzing text data from interviews and surveys to identify patterns and risks. The discussion covers continuous risk assessment facilitated by AI, reducing the time and effort required for traditional assessments. Additionally, the paragraph touches on the role of AI in scenario analysis, where data from various departments can be used to simulate different scenarios and inform risk assessments. The conversation also includes the use of AI in engagement risk assessments, risk scoring, and improving communication across departments.

🌐 Global Conversations and AI-Driven Risk Assessments

This paragraph focuses on the practical applications of AI in enhancing risk assessment processes, making them more dynamic and efficient. West Blocki and Ethan Rohani discuss the use of AI for risk evaluation, identification, and impact assessments, as well as scoring analysis. They mention the development of a tool that uses a dynamic framework model for weighted scoring based on an organization's framework, allowing for a more tailored and accurate risk assessment. The paragraph also highlights the benefits of AI in enabling global conversations at any time, improving employee satisfaction and streamlining preliminary discussions.

🛡️ Generative AI's Role in Cybersecurity

Brian Willis concludes the script by discussing the practical applications of generative AI in cybersecurity. He emphasizes the benefits of AI as a tool that provides audit accuracy and consistency, offering a conversational interface to access documented cybersecurity information. The paragraph outlines how AI can support compliance programs by providing a reliable knowledge base and reducing the cost of compliance through more efficient audit activities. Willis also mentions the ability of AI to quickly review and search through extensive documentation, thereby saving time and resources in the auditing process.

Keywords

💡AI in Compliance Programs

AI in compliance programs refers to the application of artificial intelligence to ensure that organizational processes adhere to regulatory standards. This can include the use of AI to automate and streamline compliance checks, generate reports, and ensure accuracy in compliance activities. In the script, David Petrisky and Brian Willis discuss how AI models like GPT can be trained on specific compliance documents to answer detailed questions accurately, enhancing the efficiency and effectiveness of compliance programs.

💡PCI GPT

PCI GPT is a customized AI model specifically trained on Payment Card Industry (PCI) compliance documentation. It is designed to provide accurate and informed responses to queries related to PCI requirements. In the script, Brian Willis mentions the creation of a PCI GPT at LBMC, which is trained on PCI documents, report templates, FAQs, and other supporting documents, enabling the tool to provide precise answers regarding PCI compliance requirements.

💡Continuous Risk Assessment

Continuous risk assessment involves the ongoing evaluation of risks within an organization rather than periodic assessments. This allows for more dynamic and timely identification and mitigation of risks. Canal Agrawal discusses how AI can facilitate continuous risk assessments by automatically analyzing data, identifying high-risk areas, and updating risk evaluations in real time, which helps organizations maintain a proactive approach to risk management.

💡Scenario Analysis

Scenario analysis is the process of evaluating possible future events by considering alternative possible outcomes (scenarios). It is used in risk management to predict and prepare for various potential risks. In the script, Agrawal explains how AI can enhance scenario analysis by integrating data from different departments and running simulations to provide insights for risk assessments, enabling more informed decision-making.

💡Risk Scoring

Risk scoring is the process of quantifying the level of risk associated with different aspects of an organization's operations. It helps prioritize risks and allocate resources effectively. In the video, risk scoring is mentioned as a use case for AI, where AI tools can evaluate, identify, and score risks based on the organization's framework, making the risk assessment process more efficient and accurate.

💡Generative AI

Generative AI refers to artificial intelligence systems that can generate new content, such as text, images, or code, based on the data they have been trained on. This type of AI can create responses, complete tasks, and provide information in a conversational manner. In the context of the script, generative AI tools like ChatGPT are used to support cyber security audits by quickly retrieving relevant information from extensive documentation, thus improving audit accuracy and efficiency.

💡Audit Accuracy and Consistency

Audit accuracy and consistency refer to the precision and uniformity in conducting audits, ensuring that the findings and processes are reliable and repeatable. Brian Willis emphasizes how AI, particularly generative AI, can enhance audit accuracy and consistency by providing precise answers and reducing the chances of errors. AI tools can analyze large volumes of data consistently, helping auditors maintain high standards in their evaluations.

💡Cost of Compliance

The cost of compliance includes the expenses incurred by an organization to adhere to regulatory requirements, which can involve audit time, resources, and financial expenditure. The script highlights how AI can reduce these costs by automating parts of the compliance process, such as document review and data analysis, thereby decreasing the amount of time and effort required from human auditors and lowering overall compliance costs.

💡Dynamic Framework Model

A dynamic framework model in risk assessment is a flexible and adaptable approach that allows organizations to input their specific risk evaluation criteria and weights. This model adjusts to different organizational needs and industries. West Blocki describes a tool being developed to use a dynamic framework model for risk assessment, enabling tailored risk scoring and pinpointing focus areas specific to each organization.

💡Cyber Security Audits

Cyber security audits involve the systematic evaluation of an organization's information systems to ensure they are secure and compliant with regulatory standards. AI is used in cyber security audits to enhance efficiency by quickly analyzing extensive documentation and identifying vulnerabilities. Brian Willis discusses how generative AI tools are being employed in cyber security audits to improve audit processes and ensure thorough compliance with security requirements.

Highlights

Introduction of AI's role in internal audit, risk, and cybersecurity.

David Petrisky discusses how AI models are trained to understand specific controls and operations within organizations.

Brian Willis explains the creation of a PCI GPT model at LBMC, which is trained on PCI documents to provide informed answers.

The PCI GPT model helps in obtaining specific requirements for multi-factor authentication and data encryption.

Canal Agrawal highlights the usefulness of AI in continuous risk assessment and scenario analysis.

AI's capability to analyze text data from interviews and surveys for identifying patterns in risk assessments.

AI facilitates automatic and continuous risk assessments, reducing the manual workload by 60-70%.

Use of AI in scenario analysis to integrate data from different departments for risk assessment.

AI's role in risk scoring and communication between audit teams and other departments.

West Blocki and Ethan Rohani discuss dynamic risk assessment models that use AI for risk evaluation, identification, and scoring.

AI tools enable global teams to have asynchronous conversations, improving employee satisfaction and efficiency.

Brian Willis returns to discuss the practical applications of generative AI in cybersecurity.

Generative AI tools like ChatGPT provide accurate and consistent information for cybersecurity audits.

AI reduces the time and cost of compliance by quickly reviewing large documents and finding relevant information.

AI enhances audit accuracy and consistency by providing a reliable knowledge base.

Brian Willis notes that AI supports both auditors and those responsible for maintaining compliance programs.

Transcripts

[Music]

The Institute of internal Auditors

presents all things internal audit Tech

in this episode hear from multiple

thought leaders on how AI is being used

in risk and cyber security audits

they'll discuss the opportunities in

benefits AI offers internal Auditors

first let's jump into AI in compliance

programs with David petrisky director of

Professional Standards at the IIA and

Brian Willis senior lead auditor at lbmc

have you seen use cases where people are

uh training the AI model to on their

system so the the model understands you

know the uh the controls and and the

operations in their particular

organization yes and in fact one of the

uh one of the great features with say a

chat GPT is that you can actually you

can actually create custom gpts and then

train that on specific information what

we're doing at lbmc and and you've asked

about PCI but specifically around PCI is

we developed a a a PCI

GPT and we've introduced all of the PCI

documents the report templates the uh

the FAQs the uh supporting documents the

you know knowledge based documents that

they've published uh into this tool and

based on having all based on all of that

information we're able to then prompt

that GPT with questions about hey what

are the specific requirements around

multiactor authentication or data in

encryption and we can get the answers we

need specifically around that and we can

know that because it's been trained on

that document on that PCI documentation

that the answers we're get we're getting

are well informed and it's not just uh

maybe hallucinating and just making up

answers that it's called off of uh uh

off of the internet next let's turn to

Canal agrawal director of customer

success at diligent to discuss the

usefulness of AI in continuous risk

assessment and scenario analysis how are

internal Auditors using artificial

intelligence for risk assessments so I

would say there are different areas

where internal Auditors can really find

AI to be useful number one definitely is

the interviews and surveys so AI really

gives a lot of power to analyze the text

Data uh which are part of the surveys uh

and it can create different patterns

which can flow into uh as an input into

your into your process the number two

could be um you know the automatic risk

assessment uh which means that you're

trying to get into a more uh continuous

risk assessment process uh so you're not

waiting for a certain period or certain

time frame to do your risk assessment

but you already have the highest risk

identified through Ai and then what

you're trying to do is add on whatever

you want to add to that so the 60 to 70%

of the job is already done the other

thing could be you know scenario

analysis where uh you know you can

actually bring in data from different

departments and then you can run

scenarios uh based on that to to get the

input for for your risk assessment is it

being used at all uh at an engagement

level for engagement risk assessments I

mean you mentioned the surveys and that

would probably uh be a pretty good

method but are there other ways that uh

it's being used to scope engagements or

identify risks within particular subject

areas I think it is definitely used in

Risk scoring for sure I think that is

one area where it is definitely used uh

communication is another area we're

picking up where if audit teams are

communicating with other departments

sometimes you you want to make sure that

the audit teams not only the audit teams

but the other teams outside of audit are

aware of the risk so you know it is also

helping out in communication outside of

the audit department so there's

engagement there is risk scoring and

there's communication building on that

West blocki Senior manager at Grant

Thorton and Ethan Rohani principal at

Grant Thorton highlight how AI is being

applied to enhance the risk assessment

process making it more Dynamic and

efficient are there any other uh

applications or or use cases that uh you

know you see out there that we haven't

touched on yet that you think you know

you want to get many be careful what you

asked for I could go on for hours but uh

I would say um one of the big ones that

we're working on right now the risk

assessment space okay so um there's a

lot of opportunity for risk evaluation

risk

identification um performing risk impact

assessments and um doing scoring anal

yeah will it uh uh uh forecast estimate

uh you know a risk exposure so we are

working on a tool right now that I'll

actually do that with with the a dynamic

framework model so you can actually

input your organization's framework for

for weighted scoring because every

organization is a little bit different

depending on the industry and the the

business so uh you can be able to you

can input that information and and

without giving away too much before you

roll it out uh it will allow you to to

help um score and and risk Rank and and

pinpoint areas of focus I will say one

of the most interesting use cases that

I've seen and it's related to the risk

assessment question is enabling folks to

have conversations at all hours of the

day and doing the preliminary

discussions with the AI and Gathering

that information so that when the humans

actually talk it's a much deeper more

useful conversation and you've gotten a

lot of the little things out of the way

kind of streamlines things it also

enables somebody that's in Denver

Colorado to have a conversation in

Bangalore on their time schedule so that

you're not trying to shift hours to have

a conversation at 2 in the morning so

again employee satisfaction goes

skyrocketing when you're not getting up

at 2: in the morning to go have a

conversation those global Co conference

calls yeah thank you very much for your

time thank you D appreciate it all right

thanks finally Brian Willis returns to

discuss the Practical applications of AI

in enhancing risk

assessment can you tell us a little bit

about how generative AI is being used in

cyber security yeah it's a great

question um AI really uh is presenting

itself as an as a very effective and

promising Tool uh for cyber security

audit and compliance um and particularly

when we talk about AI uh I think the

thing that that most people are are

talking about is generative AI so chat

GPT and co-pilot and tools like that um

and I think the way I like to think

about it is imagine if you could add a

team member who knew everything about

every everything that was ever

documented about cyber security audit in

compliance that's what a having AI as a

tool uh for your compliance program is

like so uh even better than uh your

traditional Google search uh where you

would uh perform a search and have to

look through links and information

everything now you can get that

information just in a conversational

manner uh so it really is a a great tool

that's benefiting our uh our industry a

couple of the key benefits I like to

talk about are audit accuracy and

consistency so just like with getting a

Google search you're able to go through

uh documented information that's been

published on the internet the same way

that's where that information that a

generative AI tool uses comes from the

straight from the internet and so when

you're having that conversation it's

like being able to get directly to that

information without having to click

through search links and things uh so it

brings that element of of accuracy

consistency it can support your program

uh again through having that reliable

knowledge base uh to be able to support

folks who are both conducting audit and

as well as those folks who have have

responsibilities for uh implementing and

maintaining a compliance program the

other benefit I like to think about are

the the cost of compliance both in terms

of audit time and expenditure um so uh

at obmc we're using already a couple of

tools to support and supplement our

audit activities to where uh the the

tool allows us to review documentation

review evidence that our clients provide

to us in a much more timely manner it

can search through a 300 page uh

security policy and find the answers

we're looking for in an instant uh

without somebody having to search

through that document likewise uh if you

are um for a team that is either

responsible for maintaining compliance

or for conducting an audit if you're an

internal or an external auditor it just

results in fewer man hours uh on the

audit you're able to uh go through these

activities execute them quicker and so

the cost of compliance uh comes down so

just a couple of key benefits that we're

seeing with AI and in cyber security

well thank you very much Brian it's been

great talking to you about uh internal

audits use of artificial intelligence if

you like this podcast Please Subscribe

and rate US you can subscribe wherever

you get your podcast you can also catch

other episodes on YouTube or at the

i.org that's T he a.org

[Music]