The AI Governance Challenge | PulumiUP 2024
Summary
TLDRIn this session, Paa Kalahan, a technical director at Charles River Laboratories and a Google Developer Expert, discusses the importance of AI governance in scaling AI across organizations. He emphasizes the need for ethical and responsible AI use, highlighting the role of governance frameworks in ensuring transparency, compliance with legal standards, and risk management. Kalahan also explores the current global landscape of AI regulations, including the EU AI Act, and outlines the stages of AI adoption within organizations, from exploration to strategic integration.
Takeaways
- 🤖 AI governance is crucial to guide the ethical, transparent, and responsible use of AI in organizations.
- 💼 AI governance ensures AI systems align with organizational values, legal standards, and societal needs.
- 🚀 AI adoption has exploded in recent years, but businesses face challenges in implementing AI effectively.
- 🌐 The global regulatory landscape is evolving rapidly, making AI governance more important than ever.
- 📜 Key AI regulations include the OECD AI principles, G20 AI principles, UNESCO's AI ethics framework, and the EU AI Act.
- 🔍 AI governance involves managing risks such as bias, unethical use, and regulatory non-compliance.
- ⚠️ Without AI governance, organizations risk reputational damage, legal issues, and inefficiency in AI projects.
- 💡 AI governance should be a multidisciplinary effort across all business units, integrating with existing governance structures.
- 📊 Effective AI governance focuses on data quality, reducing complexity, and balancing innovation with control.
- 🏢 The AI adoption journey progresses from exploration to strategic integration, with governance needed at each stage.
Q & A
What is AI governance, and why is it important?
-AI governance refers to frameworks, policies, and practices that guide the ethical, transparent, and responsible use of AI. It ensures that AI systems are developed and deployed in alignment with organizational values, legal standards, and societal benefit. It's important because it helps manage the risks associated with AI as its adoption rapidly grows.
What challenges do businesses face when adopting AI?
-Businesses face challenges such as lack of interest or knowledge among employees, coordinating AI efforts across multiple regions, managing regulatory compliance, and handling the rapid evolution of AI technologies. Without a coordinated effort, organizations struggle to adopt AI effectively.
What are the key pillars of AI governance?
-The key pillars of AI governance are AI lifecycle governance, regulatory compliance, and risk management. These components ensure that AI is used responsibly, complies with evolving regulations, and mitigates associated risks.
What could happen if AI governance is not implemented properly?
-Without proper AI governance, organizations may face damage to their reputation, legal and regulatory issues, wasted resources, inefficiencies in product development, and inability to use data effectively. It can also lead to biased or unreliable AI outcomes.
How does AI governance intersect with data governance?
-AI governance overlaps significantly with data governance, as both are crucial for ensuring the ethical, secure, and legal handling of data. Data governance ensures that AI models are built on high-quality, unbiased data, which is essential for achieving reliable AI outcomes.
What are the different levels of AI risk under the EU AI Act?
-The EU AI Act classifies AI applications into four levels of risk: (1) Unacceptable risk, which is prohibited unless authorized for national security purposes, (2) High risk, which requires conformity assessments, (3) Limited risk, which includes transparency obligations, and (4) Minimal risk, which faces no legal obligations but may adopt voluntary codes of conduct.
What was the significance of the 2019 OECD AI principles?
-The 2019 OECD AI principles were significant because they marked the first internationally adopted AI governance guidelines, focusing on ethical, transparent, and responsible AI use. This laid the groundwork for further AI governance frameworks globally.
What are some ethical concerns that AI governance addresses?
-AI governance addresses ethical concerns such as bias in AI models, lack of transparency in AI decision-making, manipulation of human behavior, surveillance, and potential discrimination in critical areas like employment, education, and access to public services.
How does AI governance help in risk management?
-AI governance helps organizations manage the risks associated with AI by implementing processes and tools that oversee the development and deployment of AI systems, ensuring that these systems align with ethical standards, legal regulations, and organizational goals.
Why is AI governance a multidisciplinary effort?
-AI governance is a multidisciplinary effort because it involves various aspects of business, including legal, technological, data management, information security, and change management. Effective AI governance requires collaboration across these different areas to ensure comprehensive oversight.
Outlines
📢 Introduction to AI Governance and the Need for Organizational Adoption
In this opening section, Paa Kalahan, a technical director at Charles River Laboratories, introduces the importance of AI governance. He outlines his roles, including being a Google Developer Expert, and sets the stage for discussing the governance of AI. Kalahan emphasizes that AI adoption in businesses requires coordinated efforts and governance frameworks to manage risks and scale effectively across multiple regions. He highlights how AI tools like ChatGPT are easily accessible, but AI adoption for businesses poses unique challenges that governance can address.
🌍 Global AI Regulations: Timeline and Key Milestones
This paragraph provides a historical timeline of significant global milestones in AI governance, starting from the adoption of the OECD AI principles in May 2019 to the EU AI Act, the world’s first comprehensive AI regulation. Kalahan highlights key events such as the G20 AI principles, UNESCO’s AI ethics framework, and the G7 generative AI risks forum. These initiatives have been crucial in setting international guidelines for AI use, with a focus on ethical, legal, and regulatory frameworks.
🚦 Categorizing AI Risks: From Unacceptable to Minimal
Kalahan explains how the EU AI Act classifies AI applications into four levels of risk: unacceptable, high, limited, and minimal. He details examples of unacceptable risk, such as AI systems used for social scoring or mass surveillance, which are prohibited unless authorized by law. High-risk AI applications must undergo conformity assessments to ensure compliance. AI chatbots and other limited-risk applications require transparency, while minimal-risk AI systems have no specific regulatory obligations but may benefit from voluntary conduct codes.
🏢 Organizational Strategies for Effective AI Governance
This paragraph delves into how organizations can implement AI governance by addressing questions about accountability, oversight, and system complexity. Kalahan emphasizes the importance of maintaining a balance between governance and innovation. He warns that excessive governance can hinder progress by introducing unnecessary administrative burdens, slowing down innovation. The key challenges include managing fast-evolving AI systems, ensuring accountability, and reducing complexity without stifling creativity or operational efficiency.
⚠️ Risks of Poor AI Governance: Legal, Financial, and Reputational
Kalahan outlines the negative consequences of lacking proper AI governance. These include potential reputational damage, legal risks due to non-compliance with emerging regulations, and inefficiencies that lead to wasted resources. He explains that without governance, organizations may struggle with inconsistent product development, misuse of data, and an inability to align AI projects with business objectives. Effective governance helps mitigate these risks by ensuring alignment with ethical practices and legal standards.
🚀 The AI Adoption Journey: Stages of Integration
This section breaks down the AI adoption journey into six stages: exploration, experimentation, adoption, expansion, systemic application, and strategic integration. Kalahan explains that organizations move from initial curiosity and pilot projects to broad AI integration across business units. At the final stage, AI becomes central to long-term strategy, driving both operational efficiency and strategic decision-making. This progression requires careful planning, regulatory alignment, and continuous innovation to ensure that AI remains scalable and effective.
Mindmap
Keywords
💡AI Governance
💡Ethical AI
💡Risk Management
💡AI Regulations
💡EU AI Act
💡Transparency
💡AI Adoption
💡Generative AI
💡Compliance
💡Multidisciplinary Effort
Highlights
AI governance is essential for ethical, transparent, and responsible use of AI across organizations.
AI adoption has exploded in recent years, especially in business, with significant challenges for organizations not fully engaged with AI.
AI governance involves frameworks, policies, and practices that ensure AI aligns with organizational values and legal standards.
AI is impacting various industries including drug discovery, fraud detection, and cybersecurity, making governance crucial.
AI's rapid growth introduces risks and necessitates governance to manage evolving global laws and regulatory standards.
The OECD AI principles in May 2019 marked a significant global effort in regulating AI before the rise of large language models.
UNESCO's 2021 AI ethics recommendation was adopted by nearly 200 member states, highlighting the global commitment to AI regulation.
The EU AI Act is the first comprehensive global AI regulation, entering into force in 2023, setting standards for risk-based AI applications.
The EU AI Act classifies AI into levels of risk from unacceptable to minimal, with certain high-risk AI uses requiring conformity assessments.
Excessive governance can stifle innovation by adding unnecessary administrative work and slowing down AI adoption.
AI governance must be multidisciplinary, involving legal, data, technical, and ethical units across the organization.
Organizations must balance the need for innovation with ethical and responsible AI use, as rapid AI evolution poses significant challenges.
Without AI governance, companies risk legal issues, wasted resources, and inefficiencies in product development.
AI governance overlaps with data governance and corporate governance, ensuring AI is responsibly integrated within the organization.
The AI adoption journey involves exploration, experimentation, adoption, expansion, systemic application, and strategic integration.
AI becomes a strategic tool when integrated into an organization's core functions and long-term goals, influencing decision-making at all levels.
Transcripts
hello everyone welcome to my session I'm
very happy to be here with you today in
this amazing conference and to talk
about the AI governance challenge uh my
name is paa kalahan I'm a technical
director at Charles River Laboratories I
am also a Google developer expert in Ai
and machine learning and I am an
appointed member of the Google developer
Advisory Board as well so yes I'm very
excited to be here today uh and to cover
this very interesting topic and let's
just
begin so uh again today I'll be
discussing why the governance of AI is
so
important why it is needed right now and
how it can help organizations to
effectively scale AI deployments across
multiple use cases and most importantly
across multiple regions and of course
um from the last couple of years uh AI
adoption has just exploded in in the
business world uh for Curious people
it's very easy to start you know playing
with AI using these tools and maybe you
know start um finding value on on the
interactions and the applications of of
uh tools like like um you know CH GPT or
Gemini and so on and uh for business
however is is a different challenge uh
maybe not everyone within a company
within an
organization uh they are interested in
learning about AI or using AI as a tool
to help them in their work and for a
business to generally uh be successful
in the AI adoption proc process a
coordinated effort is needed and this is
where H the initiative of AI governance
is is very very important so what is AI
governance so AI governance mostly
refers to Frameworks and policies and
practices that can guide the ethical
transparent and responsible use of AI
and it also ensures that AI systems are
developed and are deployed in ways that
are aligned with the organization values
uh also legal standards and of course uh
it's a aligned to um to be helpful and
useful for society and the time is now
this is a critical moment uh AI is no
longer something that is going to happen
in the future is everywhere is driving
innovation in drug Discovery from broad
detection uh cyber security in the
medical field and um a lot of use cases
that we can imagine right now they might
have an AI component into it but it's
very important for us to understand that
as AI grows and the implementation and
Adoption of AI grows so do the risks and
we are dealing with an incredibly fast
moving technology
uh
alongside rapidly evolving Global laws
and regulations and we're going to talk
a little bit about that uh so how can
organizations can start to adopt good AI
governance practice we're going to talk
about that uh today uh why should
organizations prioritize AI governance
right now and why um why should we care
right
um and I believe that we all should care
about AI governance because we are
navigating Uncharted Territory and and
with a rapid with a rapid pace of AI
development
and moving regulatory targets all around
the world that is a real real need to
ensure that we are using these powerful
technology safely responsibly and that
we uh maintain
compliance so let's have a look at
before we continue it's good to stop for
a minute and have a look at the current
landscape of you know laws and
regulations that are around the world
and how how has been like the timeline
for this so in May 2019 and we have to
be mindful that this was before the
explosion of llms and CH GPT Gemini
llama all of these models uh the
oecd uh um AI principles were adopted uh
for the first time then in June 2019 the
G20 AI principles were adopted as well
um then the
gpai which is the global partnership on
AI was launched to Foster of course
International cooperation then in
November 2021 I believe this this was a
very important um moment for the whole
uh Regulatory and and and and the the
whole worldwide effort on uh regulate
and take care of how II is being used so
the
UNESCO um recommended they they work on
this AI ethics framework and is a it was
a really important one and if you are
interested in this area I really would
recommend you to go and have a look at
this AI ethics recommendation and it was
quite significant because it was adopted
by almost 200 un uh member states from
the United Nations uh then there was a
u20 leaders declaration around the AI
regulation and pro
Innovation uh October 2023 the G7
countries um they launch a forum on the
generative AI risks so this was
important because the generative AI boom
already started and uh organizations and
governments around the world started to
um realize that it was going to be a
huge huge area of AI that um can come
with huge risks as well so um they need
to start working on on on analyzing the
risk the risks and creating of course
the the regulations around it uh on
November 2023 also there was the bled Le
declaration here in the United Kingdom
it was signed by 28 countries and the EU
as well and they during that during that
uh time they also did the they run the
first ai ai I safety
Summit then this year in March there was
a a AI resolution that was adopted by
the United Nation general assembly and
then we had uh o ecd the oecd principles
were updated from the one that was uh
created on
2019 then the council of Europe um Al
adopted an AI treaty there was a very
interesting um AI Summit in South Korea
that it was a followup on the summit
that was um run in in the in the UK in
the blery Declaration and finally what
many people believe is the the most
robust and important uh AI regulation
set of regul
regulations uh finally entered into
Force which is the EU AI act and is the
world's first comprehensive AI
regulation then they spend um a good a
good amount of years working on it they
had to adapt it to include generative AI
as well so this is the current um you
know uh the the the current landscape
that we have around uh regulations World
while and again I will highly suggest
you to go and and have a look at
it so just to have a brief overview of
specifically the euii ACT what they're
doing is that they are um analyzing the
AI applications and they are classifying
it into levels so there are it comes
from unacceptable risks all the way it
down to minimal risk so for example
certain AI applications are going to be
prohibited by law and those are the
unacceptable risk to the safety
livelihoods and the rights of people so
these use cases are prohibited unless
authoriz authorized by law for national
security purposes and that might include
uh the ones that are forbidden social is
scoring AIC s manipulation of human
behavior that might cause harm and mass
surveillance then the second level is
related to high risk uses and it will be
subject to uh a Conformity assessment
before they can be
deployed and this assessment looks at
the quality of the data and it says to
minimize the risk and discrimin
discriminatory
outcomes um so so for example access to
employment education public services and
so on then the third level is H related
to limited risk uh uses and it will only
have transparency obligation so for
example in the case of a AI based
chatboard users should be aware that
they are interacting with a AI
application and finally the last level
um includes the minimal risk excuses and
those are not going to be subject to any
obligation uh however the adoption of
voluntary codes of conduct is
recommended uh this could enhance the
trust for adoption of AI and and give
like a competitive advantage to to
service
providers right so let's start talking
about what can organizations do uh
around AI governance so again um AI
governance let's remember is a set of
rules practices processes and tools that
are employed to to
ensure uh that the organizational use of
AI Technologies aligns with the
organization's strategies the objectives
um their values and that they also
fulfill legal requirements and something
that is very important and we cannot
forget about it is that it meets the
principles of ethical and responsible
use of AI followed by the
organization
so we can start by acknowledging that
fast moving AI Innovation is putting
pressure on companies so there is a comp
competitive Advantage companies are
adopting AI uh very very in a very rapid
way to stay competitive and avoid losing
market share to AI adopting uh Rivals
then we have the Regulatory Compliance
uh governments are beginning to discuss
an enact regulations around AI adding a
layer of complexity to its adoption and
and and its use then there is the
pressure to innovate again uh companies
must quickly integrate AI or they risk
failing behind in the in in this rapidly
evolving technological landscape then we
have the ethical and governance
challenges
integrating AI into
business uh we need to be careful we
need to have a careful consideration of
ethical implications and of course how
are we going to uh how the governance of
this is going to happen and then the
evolution of uh AI
capabilities AI Technologies is going to
continue advancing very fast and
companies they need to keep up with the
latest development to um be able to take
advantage of its You full potential and
then of course a very very important
piece of this is the risk management so
companies need to have a very robust
framework to to be sure that they can
take care of the risks that are
associated with these kind of
Technologies and we're going to talk
about a little bit about this later but
uh one thing that is important to start
is that the effort of AI governance is a
multidisciplinary effort within a
business it has to be shared across all
of the business units because AI the the
impact of this is not is not only
technological a technical problem it's
also a legal problem it's also a data
problem and and so so is change
management is information security of
course so it covers a lot of different
areas so we need to be we we need to
have in mind the AI life governance the
Regulatory Compliance and the risk
management those are the three pillars
that we have to uh think about when when
we start to uh Implement an AI
governance
um strategy within your
company so when we are starting to
implement AI governance we need to ask
these questions so as the system grows
and it gets more
complex uh the the the challenge of
maintaining and updating these uh
intensifies
so um how how is this going to affect uh
these processes then who will be
responsible for having uh or overseeing
the entire AI application uh or the this
AI system who who's going to have the
overview then if something goes wrong
who is accountable accountable for
responding to to issues when when they
arise who takes actions when something
goes wrong then uh we need methods for
recognizing and and rectifying the use
of maybe incorrect data or
algorithms how will the use of wrong
data or algorithms is are going to be
detected uh then we need to think about
the approaches to enhance the
system and reduce its complexity how
will the system uh be improved and
complexity reduced and this is a very
tricky thing in AI again because AI is a
field that is evolving very very fast
and it's very hard to keep up and it's
very hard maybe you have an
implementation using
llms and maybe you need to use now a
better a better model came about maybe a
new version of the model that you
already used and changing that is is is
very complex and and we have we're
dealing with a new way of uh creating
software a new way of uh integrating
these tools into our Solutions so that
is something that we really need to
think about as
well and then the danger of excessive
control right if you implement too much
governance uh it can be actually contr
counterproductive because if we increase
the administrative work and we put more
red tapes around it more bureaucracy it
will further complicate the system so uh
adding too much governance again is
counter effective and and will only add
to the that burden and it will create
more complexity and it might slow down
innovation
so let's think about what happens when
AI governance is not in place so we can
have obviously um you know as as
exciting as AI is without proper
governance it can lead to serious issues
that can impact not just the technology
or or the solutions that we're using but
also the entire organization so first of
all damage uh uh to the organization's
reputation uh a lack of AI governance
can can damage our reputation when AI is
misused or makes harmful decisions it
can be due to bias lack of transparency
unethical practices copyright issues it
can really impact the public trust so
especially in industries that work in
the areas of Health Care Finance law
enforcements uh this is very important
because in these areas trust is
everything so then we have the legal and
Regulatory issues without governance we
expose our organization to legal and
Regulatory risks uh again as we saw
earlier governance are governments are
introducing very rapidly laws and
regulations around the use of AI and if
the if our AI system don't comply with
this evolving standards we can face
fines sanctions or even lawsuits so
these consequences are becoming more
common as AI regulations they get like
more important globally so we really
need to think about is our legal team
ready to handle this so an AI governance
strategy an AI governance initiative can
help to
organize and and and and yeah like it
can help to yeah to organize all of
these different aspects across the whole
uh organization then wasted resources if
we don't have a clear governance
framework uh the AI projects they can
become inefficient they can become
expensive we can be doing repetitive
task duplicated effort so
um maybe teams they might be spending
time building models or system that they
don't align with the company objectives
they don't align with the legal
requirements
so the risk is to be investing heavily
in projects that don't deliver
value then that brings also to
inefficiencies in developing
products uh so so governance or the
lacking governance gaps or the lacking
of of governance can also lead to this
in in in the product development without
a proper oversight AI models might be
built with the wrong data they might
fail to do the uh the work correctly uh
or or maybe just yeah not working as as
intended so this can mean this can bring
delays on bringing the products to the
market uh and sometimes just canceling
the whole projects um another
issue is the inability to use the data
for training AI models in an efficient
way so without governance um this you
might not be able to use your data
effectively um AI thrives on quality
data but if you don't have a strong
governance to ensure that the data is
handled ethically securely and legally
then you might be on a position that
you're unable to leverage the full power
of your data sets and um what's even
worse of improperly managed data can
result in bias or unreliable AI outcomes
but for that we have data governance
right so you will see how data
governance and AI governance they
overlap and and they need each
other so that's what I'm talking about
here uh as you can see in this uh graph
to have an effective AI governance we
should uh build on other areas of
corporate governance that is already
there so we avoid duplicating like the
duplication of processes so like I said
before AI governance overlaps with data
governance and that are part of the it
governance that the company probably
already has and that lies on also onto
the whole corporate governance so
um if we want to to ensure that AI
systems are developed and deployed
responsibly AI governance must be like a
core component of the
organizations like the overall
governance structure is not enough and
this is very important to manage AI in
isolation AI govern governance
needs to work in harmony with other
governance Frameworks in order for it to
be truly effective so um we need to
implement an AI governance plan uh then
supervise the AI life cycle monitor data
provenance ensure the quality of data
reduce the risk associated with the use
of AI so you can see there that we have
uh we can have the
Data Business unit uh involved in this
we can have information security involv
in this so is a it's a
effort that has to be done by many
different business units across the
organization so let's have a look at the
AI adop adoption journey and as you can
see the the journey can be broken down
into several stages and each represents
like a different level of maturity in
how II is integrated into an
organization so we can first uh it all
starts with the first stage which is
exploration so this is very fast and if
you can see this whole process it starts
really really fast and then it it will
slow down as we uh progress into the
journey so first it starts with
exploration right so in this stage
organizations are just starting to look
into AI possibilities they explore how
AI could be applied to their operations
they identify use cases they might
gather initial data and this is more
driven by curiosity and
experimentation and it's is a fast uh
movement because teams are exploring
just uh opportunities and then we go to
the second uh stage which is
experimentation so after identifying
potential use cases organizations start
running proof of concept projects or
pilots and they test AI models maybe on
a small scale on isolated environments
and they want to see how well the
technology performs if it it's deliver
if it if the solutions might deliver
meaningful results so this stage is all
about learning iterating and and gaining
like Trust on the
technology then the next step if the
experimentation phase shows positive
results we move into the adoption stage
so at this point the organization they
start to adopt AI Solutions more broadly
and AI begins to be embedded in in
certain uh maybe business units or
workflows but it's still very limited in
in
scope um so this phase of adoption is
key for building AI into the like the
foundation of the business and it really
helps to lay the groundwork for a a a
broader um
deployment so after adoption
organizations and enter the expansion
stage so here the use of AIS scales
across more Department uh
projects or or again a business units
and an AI starts driving value in in a
lot of different areas so it's just not
isolated experiments anymore so now
organizations can focus on refining thei
models improving the data pipelines and
integrating AI into a larger business uh
strategies then at this stage AI has
move into systemic application so AI
becomes a critical component of business
operations and is used is standardized
across the whole organization so what
this really means is that AI is no
longer seen as like an experimental tool
but as a regular tool tool that supports
like everyday processes and decision
making and then on the uh finally we we
reach the Strategic integration so in
this phase AI becomes an integral part
of the organization's long-term strategy
is not just use
um in operations but also helps to drive
a strategic decision making at the
highest levels so AI is embedded into
the company's core function and its
potential is fully
realized however this stage is is often
slower at because it requires alignment
with the company's long-term goals the
culture and the regulatory
Frameworks so as we can see the AI
adoption Journey starts fast during
exploration but becomes
like more methodical as AI becomes
Central to the
organization and by understanding where
you are on this journey you can plan the
necessary steps to move forward with
confidence and and and so you can ensure
that AI is both scalable and and is
aligned with your business
strategy and yes again uh thank you so
much for for having me and you can scan
that QR code over there if you want to
get in touch with me and of course I
will be around to answer uh all of your
questions and if you have further
questions about AI governance or AI uh
any AI related topic I will be more than
happy to help you so thank you everyone
for joining me and keep on enjoying this
fantastic conference thank
you a
5.0 / 5 (0 votes)