Accelerate the value of generative AI with three secret ingredients

Summary

El video ofrece una visión detallada sobre cómo las organizaciones pueden aprovechar el poder de los datos y la inteligencia artificial (IA) para transformar sus negocios. Se discuten tres componentes clave para el éxito en la implementación de soluciones generativas de IA: patrones, perfiles de usuario y controles. Los patrones se refieren a componentes comunes que promueven la reutilización y la escala, los perfiles de usuario se centran en los resultados de negocio y cómo los individuos obtienen valor de la tecnología, y los controles son esenciales para garantizar la seguridad y la confianza en la solución. Además, se exploran técnicas para la selección de modelos de IA, la ingeniería de prompts y la gestión de sesiones para mejorar la calidad de los resultados. La presentación incluye una demostración de cómo PWC ha desarrollado una plataforma para facilitar la experimentación rápida y la personalización basada en perfiles de usuario, con un enfoque en el control y la gobernanza para garantizar la seguridad y el rendimiento de las soluciones de IA a gran escala.

Takeaways

• 📈 **Estrategias de patrones**: Los patrones son componentes comunes que impulsan la reutilización y la escala, disminuyendo costos y evitando la duplicación de esfuerzos en la organización.
• 🧑 **Personas clave**: El enfoque en los resultados de negocio y en los individuos que utilizan la tecnología es fundamental para integrar y obtener el máximo valor de las soluciones generativas de IA.
• 🛡 **Controles de seguridad**: La seguridad y la conformidad son cruciales para garantizar la confianza en la implementación de soluciones de IA, requiriendo un control sistemático para auditorías y garantizar la calidad de los resultados.
• 🔍 **Búsqueda de información**: La capacidad de generar respuestas a partir de datos estructurados y no estructurados es una de las aplicaciones destacadas de la IA generativa.
• 🤖 **Evolución de bots**: Los bots de servicio al cliente y la capacidad de responder rápidamente a consultas son ejemplos de cómo la IA generativa puede mejorar la productividad y la eficiencia.
• 🌐 **Transformación de idiomas**: La traducción y el procesamiento de lenguaje natural son áreas en las que la IA generativa está teniendo un impacto significativo, mejorando la productividad del desarrollador y la comunicación.
• 📚 **Creación de contenido**: La generación de contenido de marketing, documentos, código e imágenes es una aplicación en crecimiento donde la IA demuestra su versatilidad.
• 🔢 **Modelos y tamaños**: La elección del modelo de IA correcto depende del caso de uso y del tamaño de los datos, lo que afecta el rendimiento y la precisión de la solución.
• 📈 **Escala y producción**: El desafío principal para las empresas es la capacidad para escalar y poner en producción casos de uso exitosos de IA generativa a lo largo de múltiples funciones empresariales.
• 👥 **Diversidad de roles**: La colaboración entre múltiples roles dentro de una organización es esencial para la implementación exitosa y escalable de soluciones de IA generativa.
• 📝 **Gestión de cambios**: La educación y el acompañamiento durante la implementación de nuevas tecnologías son clave para su adopción y éxito a gran escala.

Q & A

¿Quién es Sora Mohanti y qué papel desempeña en PWC?

Sora Mohanti es socio en PWC y se enfoca en la práctica de datos y AI de la empresa, pasando la mayor parte de su tiempo ayudando a organizaciones a desbloquear el poder de los datos y la inteligencia artificial para transformar sus negocios.

¿Qué áreas de la empresa están experimentando una transformación con la ayuda de la IA?

La transformación está ocurriendo en múltiples funciones empresariales, donde las soluciones de IA están siendo implementadas con éxito y luego desafiantes a escalar y producirse para aprovechar su valor y medir el retorno de inversión (ROI).

¿Cuáles son los tres ingredientes clave mencionados para soluciones generativas de IA?

Los tres ingredientes clave son patrones, personas y controles. Los patrones son componentes comunes que impulsan la reutilización y la escala; las personas se enfocan en los resultados de negocio y en qué obtienen de la solución; y los controles aseguran la seguridad y la confianza en la implementación de la tecnología.

¿Cómo son diferentes los patrones de manejo de datos no estructurados y estructurados?

Los patrones de manejo de datos no estructurados y estructurados difieren en cómo se interactúa con ellos. Mientras que los datos estructurados se convierten en comandos SQL para interactuar con bases de datos, los datos no estructurados requieren un enfoque diferente para funcionar eficazmente.

¿Qué es un modelo LLM y cómo se relaciona con la experiencia del usuario?

Un modelo LLM (Large Language Model) es un modelo de lenguaje grande que se utiliza en la generación de texto y comprensión. La experiencia del usuario está enfocada en cómo se les proporciona contexto y cómo se les permite interactuar con el modelo para lograr los resultados deseados.

¿Por qué es importante la selección del modelo LLM adecuado para una tarea específica?

La selección adecuada del modelo LLM es crucial porque determina la eficacia de la solución generativa de IA en un caso de uso específico. Un modelo no adecuado puede generar respuestas vagas o no precisas, lo que afecta negativamente la calidad y confianza en los resultados.

¿Qué es un 'prompting template' y cómo ayuda en la formación de usuarios?

Un 'prompting template' es una herramienta que guía a los usuarios a entender para quién están formulando un prompt, qué información buscan, quién será el receptor de la información y qué restricciones deben considerar en su prompt. Esto ayuda a los usuarios a ser más efectivos en la obtención de la información requerida y a ser más precisos en sus solicitudes a los modelos de IA.

¿Cómo se aborda la gobernanza y el control de riesgos al implementar soluciones de IA generativa a gran escala?

La gobernanza y el control de riesgos se abordan estableciendo un mecanismo de gobernanza que incluya roles y responsabilidades definidos, políticas de uso estrictas, capacitación y un equipo responsable de regular todas las implementaciones dentro de la organización. Esto ayuda a equilibrar los riesgos y el ROI al mismo tiempo que se promueve la innovación.

¿Qué es la 'chunking' y por qué es importante en la calidad de los resultados de los modelos de IA?

La 'chunking' se refiere a cómo se divide la información para su procesamiento por los modelos de IA. Es importante porque afecta directamente la calidad de los resultados, ya que la forma en que se 'chunkea' los datos puede influir en la precisión y la coherencia de las respuestas generadas por el modelo.

¿Cómo se implementan los controles de seguridad y calidad en la plataforma demostrada por Brad Foster?

Los controles de seguridad y calidad se implementan a través de opciones configurables por el usuario, como el tamaño de chunking, la región de ejecución, los límites de tokens, el umbral de seguridad y la selección del modelo de IA. Esto permite a los usuarios experimentar y configurar iterativamente sin la necesidad de programación constante y asegura que se mantengan estándares de seguridad y calidad en los procesos.

¿Cómo se puede mejorar la calidad de los resultados en las soluciones de IA generativa?

La calidad de los resultados se puede mejorar a través del modelado de versiones, pruebas de regresión, el control de la aleatoriedad en la selección de respuestas, la gestión de sesiones de chat y la actualización y control de versiones de datos. Estos aspectos son cruciales para mantener la precisión y confianza en las soluciones de IA a gran escala.

Outlines

00:00

📈 Introducción a la transformación de negocios con datos y AI

El primer párrafo presenta a Sora Mohanti y Brad Foster, socios de PwC, quienes se especializan en la práctica de datos y la inteligencia artificial. Explican que su trabajo gira en torno a ayudar a organizaciones a desbloquear el poder de los datos y la IA para transformar sus negocios. Comienzan a discutir observaciones en industrias en las que trabajan y presentan los tres ingredientes clave para soluciones generativas de IA: patrones, perfiles de usuario y controles.

05:00

🧩 Importancia de los patrones, perfiles y controles en la IA generativa

Este párrafo profundiza en los tres componentes clave para soluciones de IA generativa: patrones, perfiles de usuario y controles. Los patrones son componentes comunes que promueven la reutilización y la escala. Los perfiles de usuario se enfocan en los resultados de negocio y en qué obtienen los individuos que los utilizan. Los controles son fundamentales para garantizar la seguridad y la confianza en la implementación de estas soluciones. Se discuten ejemplos de patrones, como la diferencia entre datos estructurados y no estructurados, y cómo se aplican en diferentes contextos.

10:01

🔗 Desafíos y soluciones en la implementación de IA generativa

El tercer párrafo aborda los desafíos que enfrentan los clientes de PwC en la implementación de casos de uso de IA generativa a escala. Se destaca la importancia de comprender los patrones a nivel empresarial y la reutilización de componentes. Se menciona la formación de equipos centralizados para la implementación y la gestión de cambios. Además, se discute la importancia de la selección del modelo de IA, el contexto, el procesamiento de lenguaje y la ingeniería de consultas para garantizar la calidad y la seguridad de los resultados.

15:02

🛡️ Controles y gobernanza en la implementación de soluciones de IA

Este párrafo se enfoca en los controles y la gobernanza necesarios para la implementación segura y eficaz de soluciones de IA generativa. Se destaca la importancia de rastrear las entradas y resultados, el control de la calidad, la gestión de versiones de datos y modelos, y la necesidad de políticas de contenido y acceso basado en roles. Se recomienda establecer un mecanismo de gobernanza para equilibrar los riesgos y la rentabilidad de las inversiones en IA.

20:04

🚀 Demostración de la plataforma de IA generativa de PwC

El quinto párrafo ofrece una demostración de la plataforma de IA generativa desarrollada por PwC. Se muestra cómo la plataforma permite la interacción con diferentes tipos de datos y casos de uso, y cómo se pueden configurar controles y opciones de usuario sin requerir conocimientos de programación. Se destaca la importancia de la personalización basada en perfiles de usuario y la capacidad de la plataforma para adaptarse a diferentes necesidades y roles dentro de una organización.

25:05

📚 Conclusión y formas de involucrarse con PwC y Google

El sexto y último párrafo concluye la presentación, ofreciendo información sobre cómo los asistentes pueden seguir involucrándose con PwC y Google en eventos futuros. Se proporciona un código QR para acceder a más detalles sobre breakfasts, sesiones y eventos de networking. Además, se agradece la participación y se cierra la presentación con un mensaje musical.

Keywords

💡Data y AI

Datos y AI son conceptos fundamentales en el video, referidos como la clave para transformar los negocios. Se discute cómo las organizaciones pueden desbloquear el poder de los datos y la inteligencia artificial para mejorar sus operaciones. Un ejemplo en el guión es cuando Sora Mohanti menciona trabajar con organizaciones para ayudarles a transformar sus negocios mediante la utilización de datos y AI.

💡Generative AI

La Inteligencia Artificial Generativa (AI) es un tema central del video, abordado por Brad Foster como una evolución en el campo de la IA. Se destaca cómo esta tecnología está siendo utilizada con Google y sus capacidades para generar contenido, respuestas y soluciones personalizadas. En el script, se menciona que Brad ha estado involucrado en el viaje de la IA desde el inicio, y ahora se enfoca en la generativa AI y sus aplicaciones con clientes de Google.

💡Modelos de LLM

Los modelos de LLM (Large Language Models) son una parte crucial de la generative AI y son mencionados en el video como herramientas para manejar y procesar información. Son importantes para la selección del modelo correcto según el caso de uso y para garantizar la calidad de los resultados. En el guión, se destaca la importancia de elegir el modelo adecuado para cada tarea y cómo Google ofrece una variedad de modelos para elegir.

💡Seguridad y Controles

La seguridad y los controles son aspectos claves al implementar soluciones de generative AI, como se discute en el video. Son esenciales para garantizar que los procesos sean seguros y confiables, y para cumplir con los estándares de cumplimiento y calidad. En el guión, se abordan temas como el monitoreo de entradas y resultados, la trazabilidad y la aplicación de políticas de contenido.

💡Personas y Experiencia del Usuario

Las 'personas' son representaciones de los diferentes roles y usuarios finales que interactúan con la generative AI. La experiencia del usuario (UX) es fundamental para el éxito de la implementación de la tecnología. En el video, se insiste en el diseño de la solución alrededor de las necesidades y objetivos de las personas, asegurando que la tecnología sea adoptada y utilizada efectivamente en la organización.

💡Patrones y Reutilización

Los patrones son componentes comunes que promueven la reutilización y la escala en las soluciones de generative AI, como se menciona en el video. Ayudan a reducir costos y a evitar la duplicación de esfuerzos. En el guión, se habla de diferentes tipos de patrones, desde la summarización de información hasta la creación de contenido, y cómo estos son fundamentales para la eficiencia y el éxito a gran escala.

💡Desarrollo y Escalabilidad

El desarrollo y la escalabilidad son retos clave para las organizaciones que buscan implementar soluciones de generative AI. El video aborda cómo los patrones y las personas pueden ayudar a superar estos desafíos. Se destaca la importancia de la producción y la medición del ROI (Retorno de Inversión) al escalar los casos de uso y la implementación de la tecnología.

💡Gobierno y Políticas

El gobierno y las políticas son mencionados en el video como componentes esenciales para la gestión responsable de la implementación de la generative AI. Incluyen la asignación de roles y responsabilidades, la capacitación y la aplicación de políticas de uso estrictas. Estas medidas ayudan a equilibrar el riesgo y el rendimiento, y son cruciales para la implementación a gran escala.

💡Procesos de RAG

RAG (Rapid Application Generation) se refiere a los procesos de generación rápida de aplicaciones, que son importantes para la integración de la generative AI en los flujos de trabajo de la organización. En el video, se discute la importancia de adaptar estos procesos a las necesidades de cada 'persona' y caso de uso, asegurando que los resultados sean precisos y consistentes.

💡Orquestación y Experiencia del Usuario

La orquestación se refiere a la integración y coordinación de las soluciones de generative AI en los procesos reales de la organización. La experiencia del usuario es crucial para la aceptación y el éxito de la tecnología. En el video, se destaca la importancia de entender cómo los usuarios interactuarán con la tecnología y de diseñar la solución para que se integre sutilmente en sus operaciones diarias.

💡Cambio de Gestión

El cambio de gestión es un aspecto importante al implementar nuevas tecnologías como la generative AI. El video destaca la necesidad de educación y apoyo para los usuarios durante el proceso de implementación. Esto incluye la capacitación en la utilización de la tecnología y la adaptación a los cambios en los procesos y flujos de trabajo existentes.

Highlights

Sora Mohanti, a partner with PwC's data and AI practice, discusses unlocking the power of data and AI for business transformation.

Brad Foster, also a partner at PwC, shares his experience with Google's AI journey and the evolution of generative AI.

Three key ingredients for generative AI solutions are identified: patterns, personas, and controls.

Patterns refer to common components that drive reuse, scale, and cost reduction in organizations.

Personas focus on the business outcomes and the experience of individuals using AI within organizations.

Controls are essential for ensuring safety, security, and the systematic implementation of AI solutions.

Different patterns for AI solutions include summarization, deep retrieval, Q&A, transformation, augmentation, and new creation.

The importance of model selection based on the use case and the right model for the job is emphasized.

Generative AI deployment requires a combination of different business roles working together for successful scaling.

Creating centralized COEs (Centers of Excellence) or deployment teams aids in detecting patterns and managing change.

The session discusses the challenges of scaling use cases and productionizing AI across enterprise functions.

LLM (Large Language Model) selection and context grounding are crucial for accurate and consistent AI responses.

Prompt engineering is key to controlling the creativity and ensuring the right outcome from AI models.

Session history and prompt context influence AI response generation, affecting the quality and consistency of results.

Orchestration and user experience are vital when integrating AI into real processes with actual users.

Change management and education are important for proper AI adoption and usage across an organization.

Governance mechanisms are recommended for managing risks and ROI when deploying generative AI solutions at scale.

The importance of regulatory risk, data privacy, and security in AI deployments is highlighted.

PwC has developed an application to allow rapid experimentation with AI based on personas and use cases, with built-in controls and metrics.

The demo showcases how controls and configurations can be systematically implemented for different AI interactions.

Transcripts

[Music]

welcome everybody uh my name is Sora

mohanti I'm a partner with pwc's uh data

nii practice and I spend most of my time

working with organizations helping them

unlock the power of data and AI to

transform their businesses um we're

going to talk to you a little bit here

about

uh some of the things that we're

observing in the industries that we deal

with h over to you

Brad um thanks s yeah and thanks

everybody for spending a few minutes

today and your busy schedules uh hanging

out with us and listen listening to this

presentation um Brad Foster I'm a

partner at at PWC uh focus a lot on the

generative AI stuff that we're doing

with Google I've been involved with

Google uh since really they started they

had three beta products years ago app

engine and cloud storage and big query

and so been a part of AI Journey for

quite some time and it's fun to see the

next uh Evolution with with generative

Ai and what we're able to do uh with

Google customers and and and solutions

out there so we're going to spend a

little bit of time today talking about

some things we've learned I think over

the past year uh that we might uh we

think are maybe useful for for you guys

to hear about uh we want to spend some

time at the end as well for Q&A so we'll

try to leave some leave some time for

for that and pass the microphones around

so like the topic of this is really

three Secret ingredients and of course

there's more that goes into this but

just to talk about three you know

important things uh around these

generative I Sol Solutions is patterns

personas and controls and The quick

summary is patterns are really those

things that are the common components uh

that drive reuse scale can drive your

cost down reusability in the

organization prevent people from doing

things you know five different times uh

personas is really the focus on the

business outcome so the the individuals

who are using these things in the

organizations and what are they getting

out of it how do you Stitch it all

together for for ultimate value uh and

one of the nuances of of how you build

these things to do that and finally

controls right none of this is going uh

into production or going to be used

unless everybody feels safe and secure

about it and so not just about the

philosophy of those controls but how do

you start to take those controls into

systematic implementation so as you're

using it you can audit it you have

assurances and it's it's not just humans

trying to follow a paper process um but

these are these applications have this

infused uh these controls infused so for

example design patterns those are things

like Which models are using for which

which Solutions um the model tuning and

and chunking how do you do that

differently and how do you save best

practices uh data types and ingestions

for your rag uh processes Etc uh

personas what what do you do to get that

right right everything starts with the

user experience you I know we like to

talk about llma models but what's the

user experience that the end of the day

how do you give that person context

what's the agent's job what are they how

are they trying to perform how do you

get the prompt engineering the rag

processes correct for that Persona and

then controls you obviously things like

security compliance um there's managing

capacity finops aspects of this uh as

well as the accuracy and the quality of

the output so these are the things we're

going to talk about a bit today um so

jumping into the patterns area a lot of

you are probably familiar with this type

of uh designation of patterns so this

isn't an end all be all but just six

ways we look at patterns um being at

summarization so of information I want a

quick summary of that could be

structured data tell me this year's

sales could be unstructured data tell me

the point of this policy uh could be you

know deep retrieval deep retrieval and

Q&A are similar right deep retrieval how

you getting you know deeper into that

semantic nature of data and what it

means in surfacing that Q&A think of all

of the the evolution of Bots the chat

Bots customer service Bots how are you

you quickly surfacing those those

questions you have

transformation uh types of things a

simple one is obviously language which

now program you hear a lot about the

developer uh productivity and and what

we're able to do with you know

translating uh whether it's code or

whether it's actual spoken language of

augmentation uh for things like

autocomplete or test data generation

could be another uh great example of

that and finally net New Creation so you

hear things like like marketing content

creation and you know imagin and all of

the the the images and and ways it can

be used for for those types of use cases

but when you look at at a high level

it's really too high level when you get

into the details of these

implementations uh these patterns start

to come into kind of level two and level

three types of patterns and I'm not

going to drain this you know ey chart

but I'll give you a couple examples and

one simple one to start with is the

difference between unstructured and

structured data so when you're you know

interacting with structure data

converting that English base prompt into

a SQL command that has to interact with

the structured database is a different

process than asking uh that same prompt

to an unstructured data set and behind

the scenes what you have to do to make

that work effectively is different so

those are two different patterns the Q&A

seems the same up front uh but the the

back end and how you do that is

different uh another is in like net New

Creation what you're trying to create

are you trying to create documents code

images so those again it's not created

the same what you're trying to do uh you

know I'm doing images so imagin is what

I might go with versus I'm just trying

to create a new document or you know an

email to send to somebody it's different

of how it works in the back end and

another thing that we didn't put on this

slide that I think is important is uh

also the types and sizes of really the

sizes of models I think you the price

performance game is is important as well

so you'll learn for these types of

questions and use cases everything isn't

going to be solved by just one huge

model doing it all you're going to find

that you know everything you know uh is

has a price and so that whether it's

slower or whether you know the the

answers are too vague because it it

wasn't a specific of a model or you're

trying to do it on a mobile mobile

device so it needs to be be condensed

that's another thing that's that comes

into play as as we're all implementing

these things so I'll turn over to sarb

now to talk about as we see these

patterns emerge and actual the

implementations of these things how that

starts to scale yeah so one of the

challenges that we are observing that

most of the most of our clients are

facing right now is not at the use case

level um I think there's a lot of use

cases that are very successfully getting

implemented across Enterprise functions

with the organizations that we work with

or we consult with or even uh you know

talk to on a regular basis where they

have a challenge is to scale those use

cases productionize them deploy them

across multiple functions to harness the

value and then measure that Roi that

that's where most of these patterns come

into the picture because these patterns

help you understand reusability of of

components like vector databases

connectors plugins that you may have

built any other uh you know data sets

that you may have modernized both

structured or unstructured the

embeddings and so as an organization uh

you know we often tell our clients to

understand those patterns across the

Enterprise and then figure out how you

can look at reusability all the way up

to getting the cic CDs done the

deployments and then even productionize

it uh at scale um I think that's been

that's been the the biggest challenge

that most companies face around scaling

and then patterns actually help you

overcome some of those challenges well

so that's a quick uh summary of some

patterns we're going to going to move

into the personas a bit now and talk

about how to view that and in the

importance of of this gen gen Journey so

uh the other thing that we've noticed uh

is to ensure that generative AI is

actually being deployed at scale and

people are harnessing the value you

require a multiple of personas to be

able to use this in a seamless way but

also contribute to the build and the

deployment of these uh Solutions at

scale so what you'll notice here is the

roles of several different uh

individuals within the businesses that

need to come together to deploy it and

what we've observed is there are

organizations that are being very

successful by creating Coes or factory

models or um you know deployment uh

teams that are centralized and are doing

it in a centralized way across multiple

different parts of the business because

then they can detect the patterns their

personas are working together and then

that also helps with the change

management because the usage and the

user stories for each of these personas

may be slightly different so when you're

building this at scale when you're

centralizing it at scale you are

actually ensuring that your user

experiences are getting completely tied

into those personas enabling the users

to adopt it much faster across multiple

aspects of of an

organization so this next slide talks a

little bit about some specifics when

you're going to do an implementation it

all seems easy easy right when you I

just want to upload one document and ask

that one question that that seems to

work every time right because you took a

simple scenario that's not scaled that

doesn't have you know an enterprise

process that's been been run for decades

wrapped around it so uh the first is llm

model selection so what what is the use

case what are they trying to achieve and

then how do you choose the right model

for for the job and that of you know

obviously the model garden and Google's

own models like there there's just going

to be a continued evolution of that that

we all want to keep an eye and take

advantage of the next is the context so

how do you ground that Persona in what

their job is and make it specific so

that it's not trying to do everything

within you know those functions within a

CFO function there's many different

parts of a cfo's organization all that

that play different roles those need to

be embedded into the context of of that

agent so to speak the grounding and rag

processes um again very important I

think this is a hot you know probably

one of the hottest topics happening

right now is how do you ground these

models in your organization's data so

that those results are coming back you

know accurate and consistent and there's

a process around getting that right um

it can be things as well if you have two

versions of the policy which one's

correct if you upload both on accident

your people could get confusing answers

and it's a you know it's a it's a data

input problem and so getting that right

making sure you're working with those um

personas to do that is is key and

there's things like chunk sizing and

overlaps so that plays a role in the

quality of the output that you're

getting for some of these models and

again back to the patterns they're going

to start to be emerge best practices for

for these use cases prompt engineering

we're all you used to you know prompting

by now to hopefully to some extent and

you know the way you ask questions is

key uh to this and do you want to

control those prompts we've seen some

clients who are saying some of this I

want the drop- down menu like here's

what I've learned are the best ways to

prompt the specific business process and

I don't want let people get too creative

with that because I I've learned what

generates the best result response

confidence another one do you want let

it to be creative um and if it's not

quite sure it's a marketing you know uh

campaign let it be creative versus hey I

want this to come back this is a legal

situation or a compliance situation

every time it should come back with the

same exact answer based on this input of

of data that that I provided it session

history and prompt context uh for those

of you use this the bit I don't know if

you've experienced the situation where

you're asking it 20 questions and then

you kind of change directions a bit and

it comes out with a response and then

you start a new new session and you ask

ask that same question and it generates

a totally different response you're like

wait a minute I ask it the same question

it's off the same pile of of data why

did it apparently come back with two

different responses and usually the

answer is well the first time it had the

context you were building up a context

from 20 questions and 20 answers and

just like a human it thinks you're

talking about directionally now and so

it uses that as part of the response

generation versus if you start a new

session and that's the very first

question you ask it you know will come

back with you know a consistent response

every time so those are just interesting

things and then orchestration and user

experience uh as you put this into a

real process with real users are are

using this um don't forget about you

know their daily life sitting down with

them how are they going to you know

enter this copy and paste answers you

know where how are they getting the

output into the next stage of the the

process they're executing so you know

don't ignore

uh that part of it and at the end of all

of this too is the change management

aspect you know this is new for

everybody um and so how to use it

properly uh education you know the

handholding as these things are

implemented are key so these are some

things uh that we're seeing as kind of

key to consider as you're you're getting

into these

implementations the the final section

here to go over a bit around controls so

how are we making sure we're doing this

safely and and securely and and and

ensuring quality results so sorry talk a

little a little bit about what we're

seeing here yeah uh actually before I

jump into that one of the things that

when you were talking about uh prompt

engineering one of the things that I

realized has been a very successful

method that organizations have used is

what we call a prompting template which

kind of allows individuals to understand

who they're prompting for what

information are they looking for who's

going to be the receiver of the

information what are some of the

constraints that you want to consider in

your prompt so what that does is that

creates a a an environment for users to

be able to get trained in in prompting

for the right information and that

actually helps uh and and that's

interesting because now when we get into

controls one of the things that we often

get asked is um how are how are uh

organizations governing uh the the

deployment of generative AI Solutions at

scale and I think this is where some of

these tough questions get asked around

regulatory risk data privacy risk who's

using it uh the solution how do we know

that the prompts are secure how do we

know that the data is secure or how do

we know that you know people who

shouldn't have access to certain

embeddings or vector databases don't

have access or how do we ensure that we

using the most effective llm how do we

know that we are picking the right

architecture um and how do we know that

we are doing it at the right cost with

the right level of uh return on

investment I think all are great

questions uh and which is why we we and

we've done this to ourselves we've you

know highly recommend putting a

governance mechanism in process when

you're deploying generative AI solutions

to manage the risk and the ROI because

both need to be balanced and when these

deplo when these deployments are

happening at scale it's very important

to understand how the risks are getting

mitigated and what kind of Roi are you

expecting versus what do you get it's

okay to do it in the use case uh

environment it's not that difficult you

have you have you know you have

contained yourselves into maybe one or

two pods that are doing it but when you

scale it out these things become uh

difficult to handle and we've had a lot

of client situations where clients have

come back saying oh there's there's been

an issue with us uh you know from a

security standpoint or from a data

standpoint uh even quality of the data

sometimes uh is questionable so which is

why going back to that earlier

conversation around personas uh the CDO

or the data organization needs to be uh

working hand inand with the deployment

team to figure out to ensure that the

data is actually right and those

patterns are right

um the next slide yeah um some of the

core Dimensions that we and I spoke

about this a little bit uh in terms of

what you need to do to ensure that your

uh governance processes are right I

think obviously we spoke about you know

ensuring there's roles and

responsibilities there's training on

usage there are very strict usage

policies uh people uh that people

understand that people have been trained

on um there is a governance team that's

responsible for governing all of the

deployments within the organization now

some of them some of this may actually

sound cumbersome and and you may think

that it's actually stifling Innovation

but it doesn't I think it just takes a

little bit of time like a fly wheel when

you push it at the beginning you need

momentum for it to get to to for it to

run and what we've noticed is that

initial push is a little bit hard but

then once you get it running like a

machine um it actually runs really

really well and some of the best

organizations in the world are are doing

a great job at governing um uh you know

the the risk versus Roi

components over to you Brad yeah so a

couple examples and we're going to do a

bit of a demo here of how this stuff

comes to life in actuality but uh some

examples of controls uh with compliance

so prompt input uh results you know and

uh the prompts inputs themselves in the

results like are you tracking all of

that so do you know what your

organization is typing in do you know

the results that are coming out and how

do you want to make sure you're

monitoring uh that on a continuous basis

uh citations so I think you guys

probably understand what citations are

but when I get a response back how do I

know where it came from oh there were 10

documents I could have pulled from which

document did the answer come from which

paragraph in that document did it did it

come from so that again you have that

audit Trail and the Assurance of that

the result is actually accurate you have

content policies uh applied to data

curation and and those results so I

think data as always will play a key

role so is the data that you're using

the approved data sets is it been

curated uh Etc so in in what which

situations do you want to make sure

you're you're forcing and locking that

down in other situations let people you

know be creative and interact uh with

data on the data side you know the the

role-based uh access to Solutions and

features so I don't think this is going

to be a world where every body has

access to every feature and function in

this landscape and it's just a

playground you're going to give people

uh some people more capabilities and

ability to experiment and others are

going to be part of a a finished process

that they're executing that was uh that

was built by an organization uh you have

the the data for rag itself I talked

about like how are you indexing that

controlling that itself um and then

really I think another thing that's

going to come into the picture is the

data versioning so over time as your

data changes how do you go back and say

well now I have a new policy I need to

go reindex that and make sure that's the

the do you know the document that's

being used and how do I actually know

and have a audit history that oh policy

number one you know stop being used in

this year in this date and this time and

this is when policy number two is

indexed and that's where answers are

based so that as again you executing

these processes at scale for especially

regulated Industries you have you know

that data versioning uh aspect of things

and then quality um so model versioning

and regression testing so we're going to

live in a world and right the side by

side or Google already has it like I

want to test these two models um and

maybe I went from Palm to Gemini and I

want to see the results or you know

Gemini W to 1.5 or a model Garden option

in a in a Google version so there you

know I think Google's been a you know

done a great job embracing that concept

and just in letting people explore uh

that in a great rapid rapid way so

that's going to be something something

that continues uh chat I mentioned the

chat session refresh so that can impact

quality results again it's it's a bit of

a user uh experience or user air

situation versus there's something wonky

happening on the on the back end so you

need to be cognizant of that quality

Randomness selection or lockdown do you

want people to dial that temperature and

the randomness up up up or down for the

quality control and then chunking

options uh as well so how are you

chunking this data that's becoming more

and more important as people scale uh

and how you're chunking strategies and

and how you look at that so it kind of

wraps up some of the discussion parts of

this uh did want to go into a short demo

before we open in the session up to to

Q&A and so what we built as we were kind

of on our journey doing this uh at PWC

was we learned that we don't want to

build the same thing over and over for

each use case we don't need a new Q&A

front to back like let's use the same

basic uh Q&A feature and function and

then the data that's going in and the

Persona that's using it is the only

thing that's changing I don't need to

you know build something ground up so we

we we said let's try to work smarter not

harder and put something in place that

allows rapid experimentation has this

kind of persona basis um utilizes these

patterns over and over so we get scale

and then has some core controls that we

think people are going to need to start

putting into into these platforms so

that's what I'm going to show you a

little bit of a demo on here it'll be

kind of a quick run through so let me go

ahead and you guys switch over to the

the demo screen now so this is um the

homepage uh of this

application uh first thing to point out

at the bottom you have the Chicklets of

the experiences that you can engage with

so you see chat explore web explore

business insights and Gemini and so

immediately we we decided what you're

trying to do dictates again a different

kind of backend and data interaction

whether you're doing interactions with

unstructured data in this chat EXP

whether you want to index internet or or

websites and that content whether you're

looking at structured data for business

insights so big query tables CSV EXL

files uh or whether you want uh Gemini

to do like image videos audio so we can

an immediate Point here for people to

interact with um the use case based you

know this application based on the use

case that they're looking at and then

another thing that that we knew was

important is as people come into this

application that there's role-based

security we know kind of what Persona

that you are what you should you should

have access to and what you shouldn't

have access to and so as you look at at

some things that that exist are right

there's there's user

Administration uh which is just normal

role-based access privileges that you're

all used to and applications we all

interact with and then well as well as

like personas so if I want to look at

various personas for HR and develop

those and this is where again you put

the context in what is this agent trying

to do uh I want to script that I want to

control that or do I want to let a user

Define that so that you know some of the

things that we do just uh in the

beginning as well as just some things

around dashboards uh and metrics that we

we look at in terms of the usage of the

application how are people using it

what's the interaction are they they

liking the results so some kind of core

uh things that we built in from an

overall perspective and then also what

we started to find uh was that we should

give people more flexibility to manip at

as opposed to having to go in the code

all of the time and choose your chunk

size or the the region you want to run

this or the temperature start to have

settings here that somebody could could

dictate where do I want to run this

which region Google region do I want to

run it in what's the temperature output

uh token limits and those obviously keep

going up and up uh safety threshold so I

want block you know if it doesn't know

the answer do I want to block it um some

few most um the model type so I want

chat bison text bison Gemini so let

people start choosing the the the model

type I want to choose the model type do

some interactions come choose a

different model type uh do some

different interactions this gets into

the side by side and then again this

gets into like chunking sizes overlap so

let people quickly come in configure

something iterate without having to

necessarily be programmers all of the

time let business people more and more

try to to again rapidly experiment with

this technology um going back to then

you know really as you look at them

jumping into the chat uh Explorer

function here and I'll just use this

will be an example of walking through an

interaction with a a set of data so here

I would start a new session for the user

and this is going to maintain session

IDs so same thing the history so if I

want to go back to a previous session

look at those results uh if I want to

name these sessions so we're we're doing

some processes for some companies and

these are called you know control one

session this date and it's going to you

know control two so you have testers and

and other things happen you know this

where they really want to document this

well uh you can then you know download

results once you get them but this is

here in your AR personas where you

choose the Persona and again this would

be dictated on the role you are so I see

everything as I'm admin but when you log

into this application you would only see

the personas you have access to to be uh

and so in this case I'm able to choose

an HR policy it it pulls in the context

and you see some other things here of

whether you want to allow responses from

you know llm itself show videos images

randomness again all those things we

could control based on who's logging in

I can hide all those things say I don't

want them to have access to do that

actually I don't want them to even be

able to upload a document I just want

them to interact you'll see here these

are indexes so these are precured data

sets that somebody's already you know

you know done the grounding ragged a

process and now this is accessible um

for that user that you this situation a

user could upload a file so this is

where you would actually go in choose

the files from my local from Google

Drive uh the doc types and this is where

I can uh give it an index name it goes

off does the upload executes and then

that index is now available here in my

my index here so in this situation I

I'll choose a uh an index

here and uh this was an HR situation so

let's say I have somebody that's looking

at HR policies and just wants to know

hey what's the gift limit for for

clients and so it's going to go look at

that um index and say in this situation

you know 100 $ per person and there's

some

exceptions uh with the program Etc a

couple things here so first is the

citation I talked about well where did

you get this answer from like oh well it

was from this document gifts

entertainment policy June

2023 uh so I know where that's coming

from and kind of the paragraphs that

that had got on that as well as I can

give it some thumbs up thumbs down on

the responses like did I like this

response did I not like this response

and we track all that on the back end

again because if you're seeing people

put a lot of thumbs down

that could be an opportunity for

training like they're writing bad

prompts it could mean the data you know

needs to be improved uh or other things

right the context situation we talked

about the process they're going through

you know to do that U might need be

improved so uh those are things that we

look at there and again they can

download all of these results here um

and those similar things are available

as you look across uh this for for these

areas so if you were to go into the

business Insight space you'll see that

as opposed to necessarily documents you

have things like files and tables that

are that are accessible here so if I

were to click on this you know I could

see that maybe there's some big creery

data sets and other things that we'll

we'll load in here in in free form um

and you know again same all of the same

personas that exist and in those

situations you're going to um you you'll

see here like show charts I'm not going

to Doo this but uh this is where you

would do I want to see sales by quarter

for the past year and again this is

going to call the structured data

backend um for that so we just wanted to

spend a few minutes showing you guys

some of like how do these controls start

to come to life how do you start to

configure these systematically so that

again not everybody's not building this

from from ground up every single time um

and that the organization can kind of

have some guard rails for for what

you're letting people do or or not do um

so anyway I think that's really what uh

we had in terms of the demo uh let me

switch back over to the final couple

slides I think um

all right so um just a wrap up session a

couple more ways to engage with uh PWC

folks if anybody's interested here's a

QR code you can get information on uh on

any of those events for for tomorrow

breakfast and a couple sessions and a a

happy hour tomorrow uh for those who are

interested uh as well as um you know I

know Google has some some other things

uh that you can get engaged with and you

can scan that QR code for that

[Music]

information