Accelerate the value of generative AI with three secret ingredients
Summary
TLDREl video ofrece una visión detallada sobre cómo las organizaciones pueden aprovechar el poder de los datos y la inteligencia artificial (IA) para transformar sus negocios. Se discuten tres componentes clave para el éxito en la implementación de soluciones generativas de IA: patrones, perfiles de usuario y controles. Los patrones se refieren a componentes comunes que promueven la reutilización y la escala, los perfiles de usuario se centran en los resultados de negocio y cómo los individuos obtienen valor de la tecnología, y los controles son esenciales para garantizar la seguridad y la confianza en la solución. Además, se exploran técnicas para la selección de modelos de IA, la ingeniería de prompts y la gestión de sesiones para mejorar la calidad de los resultados. La presentación incluye una demostración de cómo PWC ha desarrollado una plataforma para facilitar la experimentación rápida y la personalización basada en perfiles de usuario, con un enfoque en el control y la gobernanza para garantizar la seguridad y el rendimiento de las soluciones de IA a gran escala.
Takeaways
- 📈 **Estrategias de patrones**: Los patrones son componentes comunes que impulsan la reutilización y la escala, disminuyendo costos y evitando la duplicación de esfuerzos en la organización.
- 🧑 **Personas clave**: El enfoque en los resultados de negocio y en los individuos que utilizan la tecnología es fundamental para integrar y obtener el máximo valor de las soluciones generativas de IA.
- 🛡 **Controles de seguridad**: La seguridad y la conformidad son cruciales para garantizar la confianza en la implementación de soluciones de IA, requiriendo un control sistemático para auditorías y garantizar la calidad de los resultados.
- 🔍 **Búsqueda de información**: La capacidad de generar respuestas a partir de datos estructurados y no estructurados es una de las aplicaciones destacadas de la IA generativa.
- 🤖 **Evolución de bots**: Los bots de servicio al cliente y la capacidad de responder rápidamente a consultas son ejemplos de cómo la IA generativa puede mejorar la productividad y la eficiencia.
- 🌐 **Transformación de idiomas**: La traducción y el procesamiento de lenguaje natural son áreas en las que la IA generativa está teniendo un impacto significativo, mejorando la productividad del desarrollador y la comunicación.
- 📚 **Creación de contenido**: La generación de contenido de marketing, documentos, código e imágenes es una aplicación en crecimiento donde la IA demuestra su versatilidad.
- 🔢 **Modelos y tamaños**: La elección del modelo de IA correcto depende del caso de uso y del tamaño de los datos, lo que afecta el rendimiento y la precisión de la solución.
- 📈 **Escala y producción**: El desafío principal para las empresas es la capacidad para escalar y poner en producción casos de uso exitosos de IA generativa a lo largo de múltiples funciones empresariales.
- 👥 **Diversidad de roles**: La colaboración entre múltiples roles dentro de una organización es esencial para la implementación exitosa y escalable de soluciones de IA generativa.
- 📝 **Gestión de cambios**: La educación y el acompañamiento durante la implementación de nuevas tecnologías son clave para su adopción y éxito a gran escala.
Q & A
¿Quién es Sora Mohanti y qué papel desempeña en PWC?
-Sora Mohanti es socio en PWC y se enfoca en la práctica de datos y AI de la empresa, pasando la mayor parte de su tiempo ayudando a organizaciones a desbloquear el poder de los datos y la inteligencia artificial para transformar sus negocios.
¿Qué áreas de la empresa están experimentando una transformación con la ayuda de la IA?
-La transformación está ocurriendo en múltiples funciones empresariales, donde las soluciones de IA están siendo implementadas con éxito y luego desafiantes a escalar y producirse para aprovechar su valor y medir el retorno de inversión (ROI).
¿Cuáles son los tres ingredientes clave mencionados para soluciones generativas de IA?
-Los tres ingredientes clave son patrones, personas y controles. Los patrones son componentes comunes que impulsan la reutilización y la escala; las personas se enfocan en los resultados de negocio y en qué obtienen de la solución; y los controles aseguran la seguridad y la confianza en la implementación de la tecnología.
¿Cómo son diferentes los patrones de manejo de datos no estructurados y estructurados?
-Los patrones de manejo de datos no estructurados y estructurados difieren en cómo se interactúa con ellos. Mientras que los datos estructurados se convierten en comandos SQL para interactuar con bases de datos, los datos no estructurados requieren un enfoque diferente para funcionar eficazmente.
¿Qué es un modelo LLM y cómo se relaciona con la experiencia del usuario?
-Un modelo LLM (Large Language Model) es un modelo de lenguaje grande que se utiliza en la generación de texto y comprensión. La experiencia del usuario está enfocada en cómo se les proporciona contexto y cómo se les permite interactuar con el modelo para lograr los resultados deseados.
¿Por qué es importante la selección del modelo LLM adecuado para una tarea específica?
-La selección adecuada del modelo LLM es crucial porque determina la eficacia de la solución generativa de IA en un caso de uso específico. Un modelo no adecuado puede generar respuestas vagas o no precisas, lo que afecta negativamente la calidad y confianza en los resultados.
¿Qué es un 'prompting template' y cómo ayuda en la formación de usuarios?
-Un 'prompting template' es una herramienta que guía a los usuarios a entender para quién están formulando un prompt, qué información buscan, quién será el receptor de la información y qué restricciones deben considerar en su prompt. Esto ayuda a los usuarios a ser más efectivos en la obtención de la información requerida y a ser más precisos en sus solicitudes a los modelos de IA.
¿Cómo se aborda la gobernanza y el control de riesgos al implementar soluciones de IA generativa a gran escala?
-La gobernanza y el control de riesgos se abordan estableciendo un mecanismo de gobernanza que incluya roles y responsabilidades definidos, políticas de uso estrictas, capacitación y un equipo responsable de regular todas las implementaciones dentro de la organización. Esto ayuda a equilibrar los riesgos y el ROI al mismo tiempo que se promueve la innovación.
¿Qué es la 'chunking' y por qué es importante en la calidad de los resultados de los modelos de IA?
-La 'chunking' se refiere a cómo se divide la información para su procesamiento por los modelos de IA. Es importante porque afecta directamente la calidad de los resultados, ya que la forma en que se 'chunkea' los datos puede influir en la precisión y la coherencia de las respuestas generadas por el modelo.
¿Cómo se implementan los controles de seguridad y calidad en la plataforma demostrada por Brad Foster?
-Los controles de seguridad y calidad se implementan a través de opciones configurables por el usuario, como el tamaño de chunking, la región de ejecución, los límites de tokens, el umbral de seguridad y la selección del modelo de IA. Esto permite a los usuarios experimentar y configurar iterativamente sin la necesidad de programación constante y asegura que se mantengan estándares de seguridad y calidad en los procesos.
¿Cómo se puede mejorar la calidad de los resultados en las soluciones de IA generativa?
-La calidad de los resultados se puede mejorar a través del modelado de versiones, pruebas de regresión, el control de la aleatoriedad en la selección de respuestas, la gestión de sesiones de chat y la actualización y control de versiones de datos. Estos aspectos son cruciales para mantener la precisión y confianza en las soluciones de IA a gran escala.
Outlines
📈 Introducción a la transformación de negocios con datos y AI
El primer párrafo presenta a Sora Mohanti y Brad Foster, socios de PwC, quienes se especializan en la práctica de datos y la inteligencia artificial. Explican que su trabajo gira en torno a ayudar a organizaciones a desbloquear el poder de los datos y la IA para transformar sus negocios. Comienzan a discutir observaciones en industrias en las que trabajan y presentan los tres ingredientes clave para soluciones generativas de IA: patrones, perfiles de usuario y controles.
🧩 Importancia de los patrones, perfiles y controles en la IA generativa
Este párrafo profundiza en los tres componentes clave para soluciones de IA generativa: patrones, perfiles de usuario y controles. Los patrones son componentes comunes que promueven la reutilización y la escala. Los perfiles de usuario se enfocan en los resultados de negocio y en qué obtienen los individuos que los utilizan. Los controles son fundamentales para garantizar la seguridad y la confianza en la implementación de estas soluciones. Se discuten ejemplos de patrones, como la diferencia entre datos estructurados y no estructurados, y cómo se aplican en diferentes contextos.
🔗 Desafíos y soluciones en la implementación de IA generativa
El tercer párrafo aborda los desafíos que enfrentan los clientes de PwC en la implementación de casos de uso de IA generativa a escala. Se destaca la importancia de comprender los patrones a nivel empresarial y la reutilización de componentes. Se menciona la formación de equipos centralizados para la implementación y la gestión de cambios. Además, se discute la importancia de la selección del modelo de IA, el contexto, el procesamiento de lenguaje y la ingeniería de consultas para garantizar la calidad y la seguridad de los resultados.
🛡️ Controles y gobernanza en la implementación de soluciones de IA
Este párrafo se enfoca en los controles y la gobernanza necesarios para la implementación segura y eficaz de soluciones de IA generativa. Se destaca la importancia de rastrear las entradas y resultados, el control de la calidad, la gestión de versiones de datos y modelos, y la necesidad de políticas de contenido y acceso basado en roles. Se recomienda establecer un mecanismo de gobernanza para equilibrar los riesgos y la rentabilidad de las inversiones en IA.
🚀 Demostración de la plataforma de IA generativa de PwC
El quinto párrafo ofrece una demostración de la plataforma de IA generativa desarrollada por PwC. Se muestra cómo la plataforma permite la interacción con diferentes tipos de datos y casos de uso, y cómo se pueden configurar controles y opciones de usuario sin requerir conocimientos de programación. Se destaca la importancia de la personalización basada en perfiles de usuario y la capacidad de la plataforma para adaptarse a diferentes necesidades y roles dentro de una organización.
📚 Conclusión y formas de involucrarse con PwC y Google
El sexto y último párrafo concluye la presentación, ofreciendo información sobre cómo los asistentes pueden seguir involucrándose con PwC y Google en eventos futuros. Se proporciona un código QR para acceder a más detalles sobre breakfasts, sesiones y eventos de networking. Además, se agradece la participación y se cierra la presentación con un mensaje musical.
Mindmap
Keywords
💡Data y AI
💡Generative AI
💡Modelos de LLM
💡Seguridad y Controles
💡Personas y Experiencia del Usuario
💡Patrones y Reutilización
💡Desarrollo y Escalabilidad
💡Gobierno y Políticas
💡Procesos de RAG
💡Orquestación y Experiencia del Usuario
💡Cambio de Gestión
Highlights
Sora Mohanti, a partner with PwC's data and AI practice, discusses unlocking the power of data and AI for business transformation.
Brad Foster, also a partner at PwC, shares his experience with Google's AI journey and the evolution of generative AI.
Three key ingredients for generative AI solutions are identified: patterns, personas, and controls.
Patterns refer to common components that drive reuse, scale, and cost reduction in organizations.
Personas focus on the business outcomes and the experience of individuals using AI within organizations.
Controls are essential for ensuring safety, security, and the systematic implementation of AI solutions.
Different patterns for AI solutions include summarization, deep retrieval, Q&A, transformation, augmentation, and new creation.
The importance of model selection based on the use case and the right model for the job is emphasized.
Generative AI deployment requires a combination of different business roles working together for successful scaling.
Creating centralized COEs (Centers of Excellence) or deployment teams aids in detecting patterns and managing change.
The session discusses the challenges of scaling use cases and productionizing AI across enterprise functions.
LLM (Large Language Model) selection and context grounding are crucial for accurate and consistent AI responses.
Prompt engineering is key to controlling the creativity and ensuring the right outcome from AI models.
Session history and prompt context influence AI response generation, affecting the quality and consistency of results.
Orchestration and user experience are vital when integrating AI into real processes with actual users.
Change management and education are important for proper AI adoption and usage across an organization.
Governance mechanisms are recommended for managing risks and ROI when deploying generative AI solutions at scale.
The importance of regulatory risk, data privacy, and security in AI deployments is highlighted.
PwC has developed an application to allow rapid experimentation with AI based on personas and use cases, with built-in controls and metrics.
The demo showcases how controls and configurations can be systematically implemented for different AI interactions.
Transcripts
[Music]
welcome everybody uh my name is Sora
mohanti I'm a partner with pwc's uh data
nii practice and I spend most of my time
working with organizations helping them
unlock the power of data and AI to
transform their businesses um we're
going to talk to you a little bit here
about
uh some of the things that we're
observing in the industries that we deal
with h over to you
Brad um thanks s yeah and thanks
everybody for spending a few minutes
today and your busy schedules uh hanging
out with us and listen listening to this
presentation um Brad Foster I'm a
partner at at PWC uh focus a lot on the
generative AI stuff that we're doing
with Google I've been involved with
Google uh since really they started they
had three beta products years ago app
engine and cloud storage and big query
and so been a part of AI Journey for
quite some time and it's fun to see the
next uh Evolution with with generative
Ai and what we're able to do uh with
Google customers and and and solutions
out there so we're going to spend a
little bit of time today talking about
some things we've learned I think over
the past year uh that we might uh we
think are maybe useful for for you guys
to hear about uh we want to spend some
time at the end as well for Q&A so we'll
try to leave some leave some time for
for that and pass the microphones around
so like the topic of this is really
three Secret ingredients and of course
there's more that goes into this but
just to talk about three you know
important things uh around these
generative I Sol Solutions is patterns
personas and controls and The quick
summary is patterns are really those
things that are the common components uh
that drive reuse scale can drive your
cost down reusability in the
organization prevent people from doing
things you know five different times uh
personas is really the focus on the
business outcome so the the individuals
who are using these things in the
organizations and what are they getting
out of it how do you Stitch it all
together for for ultimate value uh and
one of the nuances of of how you build
these things to do that and finally
controls right none of this is going uh
into production or going to be used
unless everybody feels safe and secure
about it and so not just about the
philosophy of those controls but how do
you start to take those controls into
systematic implementation so as you're
using it you can audit it you have
assurances and it's it's not just humans
trying to follow a paper process um but
these are these applications have this
infused uh these controls infused so for
example design patterns those are things
like Which models are using for which
which Solutions um the model tuning and
and chunking how do you do that
differently and how do you save best
practices uh data types and ingestions
for your rag uh processes Etc uh
personas what what do you do to get that
right right everything starts with the
user experience you I know we like to
talk about llma models but what's the
user experience that the end of the day
how do you give that person context
what's the agent's job what are they how
are they trying to perform how do you
get the prompt engineering the rag
processes correct for that Persona and
then controls you obviously things like
security compliance um there's managing
capacity finops aspects of this uh as
well as the accuracy and the quality of
the output so these are the things we're
going to talk about a bit today um so
jumping into the patterns area a lot of
you are probably familiar with this type
of uh designation of patterns so this
isn't an end all be all but just six
ways we look at patterns um being at
summarization so of information I want a
quick summary of that could be
structured data tell me this year's
sales could be unstructured data tell me
the point of this policy uh could be you
know deep retrieval deep retrieval and
Q&A are similar right deep retrieval how
you getting you know deeper into that
semantic nature of data and what it
means in surfacing that Q&A think of all
of the the evolution of Bots the chat
Bots customer service Bots how are you
you quickly surfacing those those
questions you have
transformation uh types of things a
simple one is obviously language which
now program you hear a lot about the
developer uh productivity and and what
we're able to do with you know
translating uh whether it's code or
whether it's actual spoken language of
augmentation uh for things like
autocomplete or test data generation
could be another uh great example of
that and finally net New Creation so you
hear things like like marketing content
creation and you know imagin and all of
the the the images and and ways it can
be used for for those types of use cases
but when you look at at a high level
it's really too high level when you get
into the details of these
implementations uh these patterns start
to come into kind of level two and level
three types of patterns and I'm not
going to drain this you know ey chart
but I'll give you a couple examples and
one simple one to start with is the
difference between unstructured and
structured data so when you're you know
interacting with structure data
converting that English base prompt into
a SQL command that has to interact with
the structured database is a different
process than asking uh that same prompt
to an unstructured data set and behind
the scenes what you have to do to make
that work effectively is different so
those are two different patterns the Q&A
seems the same up front uh but the the
back end and how you do that is
different uh another is in like net New
Creation what you're trying to create
are you trying to create documents code
images so those again it's not created
the same what you're trying to do uh you
know I'm doing images so imagin is what
I might go with versus I'm just trying
to create a new document or you know an
email to send to somebody it's different
of how it works in the back end and
another thing that we didn't put on this
slide that I think is important is uh
also the types and sizes of really the
sizes of models I think you the price
performance game is is important as well
so you'll learn for these types of
questions and use cases everything isn't
going to be solved by just one huge
model doing it all you're going to find
that you know everything you know uh is
has a price and so that whether it's
slower or whether you know the the
answers are too vague because it it
wasn't a specific of a model or you're
trying to do it on a mobile mobile
device so it needs to be be condensed
that's another thing that's that comes
into play as as we're all implementing
these things so I'll turn over to sarb
now to talk about as we see these
patterns emerge and actual the
implementations of these things how that
starts to scale yeah so one of the
challenges that we are observing that
most of the most of our clients are
facing right now is not at the use case
level um I think there's a lot of use
cases that are very successfully getting
implemented across Enterprise functions
with the organizations that we work with
or we consult with or even uh you know
talk to on a regular basis where they
have a challenge is to scale those use
cases productionize them deploy them
across multiple functions to harness the
value and then measure that Roi that
that's where most of these patterns come
into the picture because these patterns
help you understand reusability of of
components like vector databases
connectors plugins that you may have
built any other uh you know data sets
that you may have modernized both
structured or unstructured the
embeddings and so as an organization uh
you know we often tell our clients to
understand those patterns across the
Enterprise and then figure out how you
can look at reusability all the way up
to getting the cic CDs done the
deployments and then even productionize
it uh at scale um I think that's been
that's been the the biggest challenge
that most companies face around scaling
and then patterns actually help you
overcome some of those challenges well
so that's a quick uh summary of some
patterns we're going to going to move
into the personas a bit now and talk
about how to view that and in the
importance of of this gen gen Journey so
uh the other thing that we've noticed uh
is to ensure that generative AI is
actually being deployed at scale and
people are harnessing the value you
require a multiple of personas to be
able to use this in a seamless way but
also contribute to the build and the
deployment of these uh Solutions at
scale so what you'll notice here is the
roles of several different uh
individuals within the businesses that
need to come together to deploy it and
what we've observed is there are
organizations that are being very
successful by creating Coes or factory
models or um you know deployment uh
teams that are centralized and are doing
it in a centralized way across multiple
different parts of the business because
then they can detect the patterns their
personas are working together and then
that also helps with the change
management because the usage and the
user stories for each of these personas
may be slightly different so when you're
building this at scale when you're
centralizing it at scale you are
actually ensuring that your user
experiences are getting completely tied
into those personas enabling the users
to adopt it much faster across multiple
aspects of of an
organization so this next slide talks a
little bit about some specifics when
you're going to do an implementation it
all seems easy easy right when you I
just want to upload one document and ask
that one question that that seems to
work every time right because you took a
simple scenario that's not scaled that
doesn't have you know an enterprise
process that's been been run for decades
wrapped around it so uh the first is llm
model selection so what what is the use
case what are they trying to achieve and
then how do you choose the right model
for for the job and that of you know
obviously the model garden and Google's
own models like there there's just going
to be a continued evolution of that that
we all want to keep an eye and take
advantage of the next is the context so
how do you ground that Persona in what
their job is and make it specific so
that it's not trying to do everything
within you know those functions within a
CFO function there's many different
parts of a cfo's organization all that
that play different roles those need to
be embedded into the context of of that
agent so to speak the grounding and rag
processes um again very important I
think this is a hot you know probably
one of the hottest topics happening
right now is how do you ground these
models in your organization's data so
that those results are coming back you
know accurate and consistent and there's
a process around getting that right um
it can be things as well if you have two
versions of the policy which one's
correct if you upload both on accident
your people could get confusing answers
and it's a you know it's a it's a data
input problem and so getting that right
making sure you're working with those um
personas to do that is is key and
there's things like chunk sizing and
overlaps so that plays a role in the
quality of the output that you're
getting for some of these models and
again back to the patterns they're going
to start to be emerge best practices for
for these use cases prompt engineering
we're all you used to you know prompting
by now to hopefully to some extent and
you know the way you ask questions is
key uh to this and do you want to
control those prompts we've seen some
clients who are saying some of this I
want the drop- down menu like here's
what I've learned are the best ways to
prompt the specific business process and
I don't want let people get too creative
with that because I I've learned what
generates the best result response
confidence another one do you want let
it to be creative um and if it's not
quite sure it's a marketing you know uh
campaign let it be creative versus hey I
want this to come back this is a legal
situation or a compliance situation
every time it should come back with the
same exact answer based on this input of
of data that that I provided it session
history and prompt context uh for those
of you use this the bit I don't know if
you've experienced the situation where
you're asking it 20 questions and then
you kind of change directions a bit and
it comes out with a response and then
you start a new new session and you ask
ask that same question and it generates
a totally different response you're like
wait a minute I ask it the same question
it's off the same pile of of data why
did it apparently come back with two
different responses and usually the
answer is well the first time it had the
context you were building up a context
from 20 questions and 20 answers and
just like a human it thinks you're
talking about directionally now and so
it uses that as part of the response
generation versus if you start a new
session and that's the very first
question you ask it you know will come
back with you know a consistent response
every time so those are just interesting
things and then orchestration and user
experience uh as you put this into a
real process with real users are are
using this um don't forget about you
know their daily life sitting down with
them how are they going to you know
enter this copy and paste answers you
know where how are they getting the
output into the next stage of the the
process they're executing so you know
don't ignore
uh that part of it and at the end of all
of this too is the change management
aspect you know this is new for
everybody um and so how to use it
properly uh education you know the
handholding as these things are
implemented are key so these are some
things uh that we're seeing as kind of
key to consider as you're you're getting
into these
implementations the the final section
here to go over a bit around controls so
how are we making sure we're doing this
safely and and securely and and and
ensuring quality results so sorry talk a
little a little bit about what we're
seeing here yeah uh actually before I
jump into that one of the things that
when you were talking about uh prompt
engineering one of the things that I
realized has been a very successful
method that organizations have used is
what we call a prompting template which
kind of allows individuals to understand
who they're prompting for what
information are they looking for who's
going to be the receiver of the
information what are some of the
constraints that you want to consider in
your prompt so what that does is that
creates a a an environment for users to
be able to get trained in in prompting
for the right information and that
actually helps uh and and that's
interesting because now when we get into
controls one of the things that we often
get asked is um how are how are uh
organizations governing uh the the
deployment of generative AI Solutions at
scale and I think this is where some of
these tough questions get asked around
regulatory risk data privacy risk who's
using it uh the solution how do we know
that the prompts are secure how do we
know that the data is secure or how do
we know that you know people who
shouldn't have access to certain
embeddings or vector databases don't
have access or how do we ensure that we
using the most effective llm how do we
know that we are picking the right
architecture um and how do we know that
we are doing it at the right cost with
the right level of uh return on
investment I think all are great
questions uh and which is why we we and
we've done this to ourselves we've you
know highly recommend putting a
governance mechanism in process when
you're deploying generative AI solutions
to manage the risk and the ROI because
both need to be balanced and when these
deplo when these deployments are
happening at scale it's very important
to understand how the risks are getting
mitigated and what kind of Roi are you
expecting versus what do you get it's
okay to do it in the use case uh
environment it's not that difficult you
have you have you know you have
contained yourselves into maybe one or
two pods that are doing it but when you
scale it out these things become uh
difficult to handle and we've had a lot
of client situations where clients have
come back saying oh there's there's been
an issue with us uh you know from a
security standpoint or from a data
standpoint uh even quality of the data
sometimes uh is questionable so which is
why going back to that earlier
conversation around personas uh the CDO
or the data organization needs to be uh
working hand inand with the deployment
team to figure out to ensure that the
data is actually right and those
patterns are right
um the next slide yeah um some of the
core Dimensions that we and I spoke
about this a little bit uh in terms of
what you need to do to ensure that your
uh governance processes are right I
think obviously we spoke about you know
ensuring there's roles and
responsibilities there's training on
usage there are very strict usage
policies uh people uh that people
understand that people have been trained
on um there is a governance team that's
responsible for governing all of the
deployments within the organization now
some of them some of this may actually
sound cumbersome and and you may think
that it's actually stifling Innovation
but it doesn't I think it just takes a
little bit of time like a fly wheel when
you push it at the beginning you need
momentum for it to get to to for it to
run and what we've noticed is that
initial push is a little bit hard but
then once you get it running like a
machine um it actually runs really
really well and some of the best
organizations in the world are are doing
a great job at governing um uh you know
the the risk versus Roi
components over to you Brad yeah so a
couple examples and we're going to do a
bit of a demo here of how this stuff
comes to life in actuality but uh some
examples of controls uh with compliance
so prompt input uh results you know and
uh the prompts inputs themselves in the
results like are you tracking all of
that so do you know what your
organization is typing in do you know
the results that are coming out and how
do you want to make sure you're
monitoring uh that on a continuous basis
uh citations so I think you guys
probably understand what citations are
but when I get a response back how do I
know where it came from oh there were 10
documents I could have pulled from which
document did the answer come from which
paragraph in that document did it did it
come from so that again you have that
audit Trail and the Assurance of that
the result is actually accurate you have
content policies uh applied to data
curation and and those results so I
think data as always will play a key
role so is the data that you're using
the approved data sets is it been
curated uh Etc so in in what which
situations do you want to make sure
you're you're forcing and locking that
down in other situations let people you
know be creative and interact uh with
data on the data side you know the the
role-based uh access to Solutions and
features so I don't think this is going
to be a world where every body has
access to every feature and function in
this landscape and it's just a
playground you're going to give people
uh some people more capabilities and
ability to experiment and others are
going to be part of a a finished process
that they're executing that was uh that
was built by an organization uh you have
the the data for rag itself I talked
about like how are you indexing that
controlling that itself um and then
really I think another thing that's
going to come into the picture is the
data versioning so over time as your
data changes how do you go back and say
well now I have a new policy I need to
go reindex that and make sure that's the
the do you know the document that's
being used and how do I actually know
and have a audit history that oh policy
number one you know stop being used in
this year in this date and this time and
this is when policy number two is
indexed and that's where answers are
based so that as again you executing
these processes at scale for especially
regulated Industries you have you know
that data versioning uh aspect of things
and then quality um so model versioning
and regression testing so we're going to
live in a world and right the side by
side or Google already has it like I
want to test these two models um and
maybe I went from Palm to Gemini and I
want to see the results or you know
Gemini W to 1.5 or a model Garden option
in a in a Google version so there you
know I think Google's been a you know
done a great job embracing that concept
and just in letting people explore uh
that in a great rapid rapid way so
that's going to be something something
that continues uh chat I mentioned the
chat session refresh so that can impact
quality results again it's it's a bit of
a user uh experience or user air
situation versus there's something wonky
happening on the on the back end so you
need to be cognizant of that quality
Randomness selection or lockdown do you
want people to dial that temperature and
the randomness up up up or down for the
quality control and then chunking
options uh as well so how are you
chunking this data that's becoming more
and more important as people scale uh
and how you're chunking strategies and
and how you look at that so it kind of
wraps up some of the discussion parts of
this uh did want to go into a short demo
before we open in the session up to to
Q&A and so what we built as we were kind
of on our journey doing this uh at PWC
was we learned that we don't want to
build the same thing over and over for
each use case we don't need a new Q&A
front to back like let's use the same
basic uh Q&A feature and function and
then the data that's going in and the
Persona that's using it is the only
thing that's changing I don't need to
you know build something ground up so we
we we said let's try to work smarter not
harder and put something in place that
allows rapid experimentation has this
kind of persona basis um utilizes these
patterns over and over so we get scale
and then has some core controls that we
think people are going to need to start
putting into into these platforms so
that's what I'm going to show you a
little bit of a demo on here it'll be
kind of a quick run through so let me go
ahead and you guys switch over to the
the demo screen now so this is um the
homepage uh of this
application uh first thing to point out
at the bottom you have the Chicklets of
the experiences that you can engage with
so you see chat explore web explore
business insights and Gemini and so
immediately we we decided what you're
trying to do dictates again a different
kind of backend and data interaction
whether you're doing interactions with
unstructured data in this chat EXP
whether you want to index internet or or
websites and that content whether you're
looking at structured data for business
insights so big query tables CSV EXL
files uh or whether you want uh Gemini
to do like image videos audio so we can
an immediate Point here for people to
interact with um the use case based you
know this application based on the use
case that they're looking at and then
another thing that that we knew was
important is as people come into this
application that there's role-based
security we know kind of what Persona
that you are what you should you should
have access to and what you shouldn't
have access to and so as you look at at
some things that that exist are right
there's there's user
Administration uh which is just normal
role-based access privileges that you're
all used to and applications we all
interact with and then well as well as
like personas so if I want to look at
various personas for HR and develop
those and this is where again you put
the context in what is this agent trying
to do uh I want to script that I want to
control that or do I want to let a user
Define that so that you know some of the
things that we do just uh in the
beginning as well as just some things
around dashboards uh and metrics that we
we look at in terms of the usage of the
application how are people using it
what's the interaction are they they
liking the results so some kind of core
uh things that we built in from an
overall perspective and then also what
we started to find uh was that we should
give people more flexibility to manip at
as opposed to having to go in the code
all of the time and choose your chunk
size or the the region you want to run
this or the temperature start to have
settings here that somebody could could
dictate where do I want to run this
which region Google region do I want to
run it in what's the temperature output
uh token limits and those obviously keep
going up and up uh safety threshold so I
want block you know if it doesn't know
the answer do I want to block it um some
few most um the model type so I want
chat bison text bison Gemini so let
people start choosing the the the model
type I want to choose the model type do
some interactions come choose a
different model type uh do some
different interactions this gets into
the side by side and then again this
gets into like chunking sizes overlap so
let people quickly come in configure
something iterate without having to
necessarily be programmers all of the
time let business people more and more
try to to again rapidly experiment with
this technology um going back to then
you know really as you look at them
jumping into the chat uh Explorer
function here and I'll just use this
will be an example of walking through an
interaction with a a set of data so here
I would start a new session for the user
and this is going to maintain session
IDs so same thing the history so if I
want to go back to a previous session
look at those results uh if I want to
name these sessions so we're we're doing
some processes for some companies and
these are called you know control one
session this date and it's going to you
know control two so you have testers and
and other things happen you know this
where they really want to document this
well uh you can then you know download
results once you get them but this is
here in your AR personas where you
choose the Persona and again this would
be dictated on the role you are so I see
everything as I'm admin but when you log
into this application you would only see
the personas you have access to to be uh
and so in this case I'm able to choose
an HR policy it it pulls in the context
and you see some other things here of
whether you want to allow responses from
you know llm itself show videos images
randomness again all those things we
could control based on who's logging in
I can hide all those things say I don't
want them to have access to do that
actually I don't want them to even be
able to upload a document I just want
them to interact you'll see here these
are indexes so these are precured data
sets that somebody's already you know
you know done the grounding ragged a
process and now this is accessible um
for that user that you this situation a
user could upload a file so this is
where you would actually go in choose
the files from my local from Google
Drive uh the doc types and this is where
I can uh give it an index name it goes
off does the upload executes and then
that index is now available here in my
my index here so in this situation I
I'll choose a uh an index
here and uh this was an HR situation so
let's say I have somebody that's looking
at HR policies and just wants to know
hey what's the gift limit for for
clients and so it's going to go look at
that um index and say in this situation
you know 100 $ per person and there's
some
exceptions uh with the program Etc a
couple things here so first is the
citation I talked about well where did
you get this answer from like oh well it
was from this document gifts
entertainment policy June
2023 uh so I know where that's coming
from and kind of the paragraphs that
that had got on that as well as I can
give it some thumbs up thumbs down on
the responses like did I like this
response did I not like this response
and we track all that on the back end
again because if you're seeing people
put a lot of thumbs down
that could be an opportunity for
training like they're writing bad
prompts it could mean the data you know
needs to be improved uh or other things
right the context situation we talked
about the process they're going through
you know to do that U might need be
improved so uh those are things that we
look at there and again they can
download all of these results here um
and those similar things are available
as you look across uh this for for these
areas so if you were to go into the
business Insight space you'll see that
as opposed to necessarily documents you
have things like files and tables that
are that are accessible here so if I
were to click on this you know I could
see that maybe there's some big creery
data sets and other things that we'll
we'll load in here in in free form um
and you know again same all of the same
personas that exist and in those
situations you're going to um you you'll
see here like show charts I'm not going
to Doo this but uh this is where you
would do I want to see sales by quarter
for the past year and again this is
going to call the structured data
backend um for that so we just wanted to
spend a few minutes showing you guys
some of like how do these controls start
to come to life how do you start to
configure these systematically so that
again not everybody's not building this
from from ground up every single time um
and that the organization can kind of
have some guard rails for for what
you're letting people do or or not do um
so anyway I think that's really what uh
we had in terms of the demo uh let me
switch back over to the final couple
slides I think um
all right so um just a wrap up session a
couple more ways to engage with uh PWC
folks if anybody's interested here's a
QR code you can get information on uh on
any of those events for for tomorrow
breakfast and a couple sessions and a a
happy hour tomorrow uh for those who are
interested uh as well as um you know I
know Google has some some other things
uh that you can get engaged with and you
can scan that QR code for that
[Music]
information
Browse More Related Video
Oracle Generative AI Strategy Update
El Peligro de la Inteligencia Artificial | Tecnología de miedo | Documental
Copilot llega al PC: ¡GPT-4o + IA en local + NUEVO HARDWARE! 🤯 Microsoft BUILD
7 herramientas de Inteligencia Artificial para crecer tu negocio
Qué es Inteligencia Artificial Generativa?
Juan Ignacio Moreno - Head of AI Strategy - INNOVA TSN
5.0 / 5 (0 votes)