From Talk to Action: How LLMs Can Act in the Real World

Summary

В этом видео представлены концепция и использование автономных агентов на основе больших языковых моделей (LLMs). Автор проиллюстрировал, как агенты могут преодолевать ограничения LLMs и применяться для решения реальных задач, таких как обработка запросов клиентов. Были представлены примеры и демонстрации, включая использование Python и инструментов машинного обучения для создания и улучшения стабильности агентов. Видео также обсуждает важность тестирования и адаптации агентов для их успешного развертывания в реальном мире.

Takeaways

• 🤖 Автономные агенты - это фундаментальный аспект больших языковых моделей, которые могут взаимодействовать с реальным миром.
• 📈 LLMs (большие языковые модели) удивительны тем, что они не только генерируют текст, но и выполняют широкий спектр задач обработки текста.
• 🚀 Агенты могут использовать внешние инструменты, что позволяет им преодолеть ограничения, с которыми сталкиваются LLMs.
• 🔧 Метод 'react' - это конкретный пример реализации агента, который использует структурированный подход для выполнения задач.
• 🔄 Процесс работы агента включает в себя цикл из трех частей: анализ текущей ситуации, выбор инструмента и наблюдение за результатами использования инструмента.
• 🛠️ Создание агента требует определения инструментов и написание Python-функций для выполнения различных действий.
• 📚 Примеры использования агентов включают обслуживание клиентов интернет-провайдера и генерацию ответов на основе документации.
• 🔒 Для обеспечения надежности агентов необходимо внимательно тестировать и улучшать их, а также ограничивать доступ к инструментам в соответствии с контекстом.
• 🔄 Использование инструментов, таких как LMQL и функционирование OpenAI, помогает снизить риски форматных и существенных ошибок у агентов.
• 🔄 Многоагентный подход может быть полезным для решения сложных задач, где различные агенты могут иметь специальные навыки и использоваться другими агентами для выполнения задач.

Q & A

Какие возможности автономных агентов в контексте больших языковых моделей?

Автономные агенты позволяют большим языковым моделям (LLMs) преодолевать свои ограничения, такие как невозможность взаимодействия с реальным миром. Агенты могут использовать инструменты и принимать решения, основываясь на текстовых запросах, что позволяет им выполнять действия от имени пользователя.

Какие примеры использования автономных агентов были приведены в презентации?

Два примера использования автономных агентов: обработка запросов клиентов интернет-провайдера (например, сброс пароля, диагностика интернет-соединения) и создание ответов на основе набора документов (например, генерация ответов по вопросам на основе текста документации).

Какие ограничения имеют большие языковые модели (LLMs)?

Ограничения LLMs включают невозможность знания о событиях после определенной даты, трудности с выполнением определенных задач (например, математических операций) и то, что они могут возвращать только текстовые ответы без воздействия на реальный мир.

Какие основные компоненты включает в себя метод реализации агента, описанный в презентации?

Метод реализации агента включает в себя написание промпта с тремя элементами: запросом пользователя, описанием доступных инструментов и структурой текста для генерации. Агент работает в цикле, анализируя ситуацию, выбирая инструмент и параметры, а затем做着 observation.

Какие инструменты можно использовать для увеличения надежности автономных агентов?

Для увеличения надежности автономных агентов можно использовать мощные модели (например, GPT-3.5 или GPT-4), проводить комплексные и реалистичные тесты, использовать фреймворки для ограничения текстовой генерации (например, LMQL) и функциональность function calling, а также адаптировать набор доступных инструментов в соответствии с контекстом.

Какие функции были объявлены в рамках презентации для улучшения взаимодействия с автономными агентами?

В рамках презентации были объявлены функции для получения текущей погоды и времени. Это было продемонстрировано с помощью функции function calling, которая позволяет модели выполнять несколько функций одновременно в ответ на запрос.

Какие ресурсы были предложены для помощи в реализации агентов на основе LLMs?

Были предложены ресурсы, такие как LLM starter kit, демонстрирующий использование LMS с примерами и подробной документацией, а также блог-пост о реализации агентов. Для клиентов Datadog предоставляется возможность получения помощи от менеджеров или автора презентации.

Какие преимущества имеет использование Python-библиотек, таких как LChain, для реализации агентов?

Python-библиотеки, такие как LChain, обеспечивают гибкость и простоту реализации агентов. Они позволяют определить инструменты и использовать общие шаблоны для прерывания текстовой генерации и обработки запросов без необходимости написания сложного кода.

Какие аспекты проекта можно повторно использовать для собственных случаев использования?

Большая часть Python-кода из проекта можно повторно использовать, заменив инструменты на собственные. Также можно использовать визуальные рецепты для выполнения расчетов без написания кода.

Как обрабатывать многоэтапные или сложные запросы с помощью автономных агентов?

Для обработки многоэтапных или сложных запросов автономные агенты могут использовать серию инструментов и определить последовательность действий на основе контекста и предыдущих действий. Это может включать отмену и перепланирование мероприятий или выполнение последовательности вопросов и ответов.

Какие инструменты можно использовать для мониторинга и анализа работы автономных агентов?

Для мониторинга и анализа работы автономных агентов можно использовать инструменты, такие как MLFlow, для отслеживания экспериментов и показателей эффективности, а также для визуализации траектории агента и оценки используемых инструментов.

Outlines

00:00

🤖 Введение в генеративные AI и автономные агенты

В этом разделе представлен обзор генеративных AI и автономных агентов, а также их применение в реальном мире. Рассматриваются возможности и ограничения больших языковых моделей (LLMs), а также способы преодоления этих ограничений с помощью агентов. Основная цель презентации - убедить аудиторию в важности и эффективности автономных агентов.

05:01

🛠️ Как автономные агенты преодолевают ограничения LLMs

Раздел описывает, как автономные агенты могут использовать инструменты для преодоления ограничений LLMs. Приводятся примеры использования агентов в службе поддержки клиентов интернет-провайдера, а также метод реакции (react) для обработки запросов и выполнения действий. Основное внимание уделяется структуре текста, генерируемого агентом, и способам обработки результатов действий.

10:03

🌐 Примеры использования агентов в веб-приложениях

В этом разделе представлены примеры использования агентов в веб-приложениях для решения различных задач. Описывается создание фиктивного веб-приложения на основе питон-функций, которые отвечают за различные инструменты, и демонстрация работы агента в контексте этих инструментов. Рассматриваются также другие примеры использования агентов, включая генерацию ответов на основе документов и использование семантических поисков.

15:05

🔧 Улучшение надежности и безопасности автономных агентов

Раздел посвящен методам улучшения надежности и безопасности автономных агентов. Обсуждаются проблемы форматирования и существенных ошибок, а также способы их предотвращения. Рассматриваются инструменты и подходы, такие как использование мощных моделей, тестирование, определение ограничений для текста, адаптация набора инструментов в соответствии с контекстом и включение человека в цикл для проверки действий агента.

20:06

📋 Ресурсы и рекомендации для создания автономных агентов

В заключительном разделе предлагаются ключевые сообщения и рекомендации по созданию автономных агентов. Подчеркивается низкий порог входа для создания первого агента благодаря общедоступным инструментам и библиотекам, таким как L chain. Представлены ресурсы, такие как демонстрационный проект, руководство по агентам и блоговые статьи, а также контактная информация для получения помощи от команды разработчиков.

25:10

💬 Ответы на вопросы и обсуждение возможностей

В этом разделе авторы отвечают на вопросы аудитории, касающиеся использования пакетов и инструментов, остановки генерации текста, многоагентных фреймворков, мониторинга производительности, подготовки документов для ретривал генерации, а также возможностей использования платформы для анализа запросов и категоризации данных. Обсуждаются также последние разработки Open AI и их влияние на будущее реализации агентов.

Keywords

💡автономные агенты

Автономные агенты - это программы, которые могут выполнять задачи без непосредственного участия человека. В видео они описываются как способ преодоления ограничений больших языковых моделей (LLMs) и выполнения действий в реальном мире. Примером является агент, который обрабатывает запросы клиентов интернет-провайдера и выполняет соответствующие действия, такие как сброс пароля или диагностика интернет-соединения.

💡генеративные AI

Генеративные AI - это тип искусственного интеллекта, который способен создавать новые данные или контент на основе обучения на большом наборе данных. В контексте видео, генеративные AI используются для создания текста, обработки языка и взаимодействия с агентами в реальном мире.

💡большие языковые модели (LLMs)

Большие языковые модели (LLMs) - это сложные искусственные нейронные сети, способные обрабатывать и генерировать текст на основе анализа больших объемов текстовых данных. Они используются для выполнения различных задач, таких как текстовая классификация, генерация текста и взаимодействие с агентами. Однако они имеют свои ограничения, такие как невозможность знать о последних событиях или информации, которая не была включена в их обучение.

💡zero shot в контексте

Zero shot в контексте - это метод обучения искусственного интеллекта, при котором модель может выполнять задачи без предварительной подготовки или обучения на соответствующих данных. В видео этот термин используется для описания способности LLMs выполнять задачи обработки текста, не имея примеров в(prompt), но понимая контекст и логическую последовательность.

💡few shot в контексте

Few shot в контексте - это метод обучения искусственного интеллекта, при котором модель предоставляется несколькими примерами в prompt для выполнения задачи. Это помогает улучшить результаты по сравнению с zero shot в контексте, так как модель получает дополнительную информацию для обучения.

💡реактивный метод

Реактивный метод - это подход к программированию и проектированию систем, при котором агент реагирует на изменения в окружающей среде,而不是主动寻求 действия. В видео реактивный метод используется для описания того, как агенты могут выполнять задачи на основе входящих запросов, анализа ситуации и выбора соответствующего инструмента для выполнения.

💡цикл обработки

Цикл обработки - это последовательность этапов, которые проходит информация или запрос в процессе обработки. В контексте видео, цикл обработки описывает структуру взаимодействия агента с LLM, включая анализ ситуации, выбор инструмента и выполнение действия.

💡функция вызова

Функция вызова - это механизм, позволяющий определить предварительно конфигурированные функции, которые могут быть вызваны моделью в процессе генерации текста. В видео используется для описания нового функционала OpenAI, который позволяет вызывать несколько функций одновременно, упрощая взаимодействие с агентами.

💡LML

LML (Language Model Library) - это библиотека, предназначенная для работы с языковыми моделями и агентами. Она позволяет упростить процесс создания и развертывания агентов, облегчая разработку и поддержку приложений на основе LLMs.

💡расширенная генерация

Расширенная генерация - это метод генерации текста, при котором модель использует дополнительную информацию, такую как предыдущий контекст или предоставленные документы, для создания более точных и контекстуально подходящих ответов. В видео это обсуждается в контексте использования агентов для генерации ответов на основе документов.

💡минимум

Минимум - это наименьшее количество, необходимое для достижения определенного результата. В контексте видео, минимум описывает использование минимальных усилий для создания агентов, благодаря доступным инструментам и библиотекам, таким как LML.

💡робастность

Робастность - это способность системы или модели сохранять стабильность и эффективность при наличии ошибок или неожиданных ситуаций. В контексте видео, обсуждается важность создания робастных агентов, которые могут корректно реагировать на ошибки и не допускать серьезных последствий.

Highlights

Introduction to the concept of autonomous agents and their potential to overcome limitations of large language models (LLMs).

Explanation of why agents are needed and the limitations of LLMs that they can help address.

Demonstration of how agents work through two practical demos.

Discussion on making agents more robust, an essential consideration for real-life deployment.

The capability of LLMs in text generation and performing a wide range of text processing tasks.

The concept of zero-shot in-context learning and few-shot in-context learning.

Limitations of LLMs, such as ignorance of recent events, difficulty with certain tasks, and lack of real-world impact.

The idea of using tools to overcome LLM limitations, similar to how humans use tools to overcome their own limitations.

A concrete example of implementing an agent in a customer service scenario for an internet provider.

The method called 'react' for implementing an agent and its simple yet effective nature.

The importance of structuring the prompt correctly to achieve the desired outcome with an agent.

The potential for agents to be used in a variety of real-life use cases and their adaptability.

The use of frameworks like lmql and features like function calling to reduce the risk of format errors in agents.

Strategies for mitigating the risks of substantial errors in agent actions.

The low barrier to entry for building the first agent and the availability of resources for getting started.

The potential for multi-agent frameworks and the benefits of having specialized agents working in tandem.

The use of platforms like Datadog for performance monitoring and analytics of agent deployments.

The simplicity of preparing documents for retrieval-augmented generation with modern tools.

The availability of guides and resources for implementing the examples shown in the presentation.

The impact of recent developments from OpenAI on the implementation of agents and functions.

Transcripts

all right hi guys welcome today's to

today's generative AI boot camp session

from talk to action how large language

models can act in the real world we'll

give everyone a couple of seconds to

kind of trickle in um just a few

housekeeping notes uh you're welcome to

use the chat function but please leave

your questions in the question mark

section on the right at the end of the

today's session there will be a short

Q&A um Vivian I'm going to go ahead and

pass it over to

you thank you Joy and good morning good

afternoon everyone I'm Vian I'm director

of AI Consulting at theu and today I'm

delighted to talk to you

about let me just share my

screen okay it's

coming

yeah so I'm delighted to talk to you

today about the notion of autonomous

agents so in my opinion it's the most

fascinating topic when it comes to large

language models and my personal

objective for this presentation today is

to convince you or at least some of you

that it's indeed the

case so I will start by explaining why

we need agents in the first place and

what limitations of llms this agents can

help

overcome then I will show you how agents

actually work and I will illustrate this

with two

demos and I will finally talk about

making agents more robust uh which is an

essential consideration for deploying

these agents in real life and then we

can have a Q&A session based on the

question that you can start adding in

the Q&A

section okay let's start with the basics

so so we can probably all agree that

llms are very impressive so first they

are excellent at text generation so if

you provide a prompt like this one they

can suggest a coherent and credible text

that expands on this on this prompt so

it can look like this for example but so

far it's not really a surprise because

that's exactly what they have been

trained for but what is a bit more

intriguing is that you can do more than

Simple Text generation you can actually

perform a wide range of text processing

tasks and the trick here is to write a

prompt such that the expected answer is

exactly the most logical continuation of

the prompt so for example here imagine

that you want to perform a

text classification task uh and in which

you want to detect the sentiment of a

product review you can just describe the

task and the input text in the prompt

and then you are most likely to get a a

relevant answer so here we would call

this zero shot in context learning in

context because we add all the necessary

information in directly in the

prompt and you can also add a few

examples in the prompt and this helps

get better performance and in this case

we talk about few shot in context

learning so all this is very useful for

plenty of real life use

cases but at the same time llms also

suffer from some significant

limitations and I'm I'm sure that you

have already experienced some of these

limitations by interacting with

jgpt so first llms can know at most what

they have been shown during their

training so their training is mostly

based on internet

content um up to to certain date this

means that llms like the one powering

chpt for example are very likely to

ignore recent events or non-public

information then llms are quite bad at

certain tasks which don't seem very

difficult for us but for example if you

simply ask CH GPT to add two big numbers

it will probably fail and you can also

ask jgpt to for example provide 10 words

with exactly 15 letters and it's likely

to fail for at least some of these words

and finally and maybe it's the most

obvious

limitation you can send some text to an

llm and you can get some text in return

and this can be nice and can be useful

but it does not have any impact on the

real world so ideally would like AI

assistants not only to take part in

conversations but also to take actions

on our behalf in in the real

world so how can we overcome these

limitations so actually we can do

exactly as we do um we as human beings

when we Face our own

limitations and what we do we use tools

so for example if I don't have a very

good memory I can use a pen and a

scratch pad if I don't see very well I

can use a pair of glasses if I don't

know something I can use Wikipedia or I

can use an internet search engine and it

can be the same for llms we can give

them access to external tools and let

them decide whether and how they will

use these

tools and in this situation the llm is

referred to as an

agent so let me give you a more concrete

example so let's imagine that you're in

charge of the customer service of an

internet provider so you have plenty of

customers sending requests and questions

for example there is something wrong

with my internet connection can you

please help me or I forgot my password

or I'd like to sign up to a certain

option so you have all these requests

from customers and at the same time you

know technically how to reset a password

how to activate an option for customer

how to perform some diagnostics for an

internet connection

problem so you have all these tools at

your disposal but here the challenge is

not to actually perform this VI actions

but to be able to properly interpret the

requests of the customers which are

expressed in natural language and

English and you also need to plan the

right sequence of

actions so how can we do this um the

answer is that it's actually quite

simple um I'm I'm going to show you now

one of the methods to implement an agent

and this method is called

react let's see how it works so imagine

that we want to handle the the request

of a customer so for example here a

customer sends us a message uh I have a

problem with my internet connection can

you please solve this what we do is that

we first write a prompt which includes

three

elements so of course we'll include the

request of the customer a and

blue but we also include a description

of the various tools that the llm can

use and we will instruct the llm to

follow certain structure for the text it

will generate so the structure is the

loop with three parts at each iteration

of the loop the first part is a thought

which is an analysis of the current

situation the second part is the choice

of a tool and the choice of the

parameters to use the tool and the last

part is an observation which is the

result of the use of the

tool so we write this prompt and then we

ask the lln the only thing that it can

really do which is generating the

continuation a completion of the

promp so that's what we do and if we

have done our job well and if our prompt

is Well

written then the text generated will

follow the prescribed format so we'll

have first a

thought then we'll have an action

requested by the

llm so here it would be uh run

Diagnostics um and then the llm will

generate the word observation and here

we know that we need to

stop um because otherwise the LM will

keep going and it will simply

hallucinate the result of the action of

course the llm cannot know the outcome

of this action because it's not

connected to our it

system so it cannot know what would be

the result here of certain diagnos of

certain

Diagnostics so we stop the text

generation and now we can pass the

purple

segment um and we can identify the

action that was requested so here when

Diagnostics we can identify the input

parameters of this action so here the

input parameter would be the ID of the

customer which is

1223 and then we can perform this action

independently from the llm using our own

internal API of our of our it

system and once we get the result of

this action like this we can append it

to all the text generated so far and we

can send all this the green part the

blue part the purple parts we can send

this again to the

llm and then the llm will just keep

going so again it will generate a

thought an action it will write

observation and here again we stop the

text generation we extract the requested

action we identify the parameter of this

action we per from the action outside of

the llm we get a result and then we

append it to all the texts generated so

far and we do this again and again until

we get an answer for our question and

that's it so you see it's quite

simple and the crazy thing that just

doing this works pretty

well and here there

are when you look at this method there

are important things to note so the

first one is that all the details

specific to your use

case um in particular everything related

to the tools that you make available to

the agent all these details are included

in the prompt they are not in the model

they are in the prompt this means this

mean that you don't need an llm that was

specifically trained to implement this

method or to implement your specific

toolbox

you just need a model that is powerful

enough to accomplish this task so you

can use a another the Shelf model you

don't need to train or to fune

anything and then um what is also

interesting that the method is quite

generic so you see that all the logic

the template for your prompt um the way

you interupt the text generation the way

you part the message all this does not

depend on your um on your tools on your

specific on the specific details of your

use case so it mean that you can reuse

all this logic from one use case to

another you don't need to develop it

again every time and actually there are

some implementation that you can

directly use in popular python libraries

like Lun chain so you don't even need to

code this so at the end of the day the

only thing that you need to specify for

your uh on your case on your case is the

definition of the tools and now we're

going to to see how it works so how do

we specify these tools so we need three

things first we need a name and a

description to explain in the initial

prompt what tools are available to the

LM the name is also used during the the

loop

uh to identify the actions that are

requested by the

LM and finally you need some code to

perform the corresponding action when

requested by the

LM but in fact if you are able to write

a python function to perform this action

in fact you already have a name a

description and the right code so for

the name you can just reuse the name of

the

function um for the descript ion you can

take the documentation of your function

and um of course you can directly and

the function itself can be the code that

is triggered by the

llm so at the end of the day the only

thing you need to implement an agent is

to be able to write python functions for

all the actions that need to be

performed by the llm so it's not

difficult at

all okay let's now see an agent in

action with

demo uh so I

created um let me show you I created in

datu several python functions that

correspond to uh the various tools of

the internet provider in this example so

all this is a fictitious use case but

basically here you can see that I

defined some functions so there is one

to reset pass the password of the

customer want to schedule the visit by

technician you can

run some Diagnostics on an internet

connection you can sign up to an option

Etc um and now I can show you a web app

that implements an agent on the basis of

these tools so it looks like this on the

left you can see the code on the right

you can see the resulting web app and um

I won't go I won't go through the code

in detail but the only thing that you

need to remember that it's generic so

here this is not specific to this to the

particular collection of tools that I

make available to the agent

here um so if you had to to change the

tools you could just reuse the same the

same

code so now here let me try an examples

I will say that here I have a problem

with my internet

connection do something

see the results that we

get okay should take a few seconds yeah

so here you can see that the the agent

uh chose a few actions so first it

decided to run some Diagnostics on the

internet connection it got the answer

here and based on this outcome of the

first action it decided as the second

action to organize the visit by

technician and then it understood that

it was enough that there were there were

no actions anymore that were necessary

and then it drafted the following

message let me show you another example

so now imagine that we have organized

this visits with the technician and

now um for example we can can say okay

I'm not

available anym

uh for the

visit by technician on

Monday and we please rky

go okay let's see how it

goes so again we see that the sequence

of actions that were determined by the

agent is quite logical so first it

cancels the appoint

M uh it gets the confirmation of the

consolation and then it knows that it it

needs to schedule a new

appointment and that's what it does and

then it know that it's uh that it's over

and it can draft this answer which is

relevant given the

context so in in this example it's

interesting

because in uh in the way we Define the

function we don't have a function to

reschedule an existing meeting we have

just given a series of tools and two of

these tools are uh how to cancel an

appointment and how to plan another

appointment but we never had explicitly

to explain to the agent that

rescheduling corresponds to cancelling

and to plan a new meeting uh that's what

the the model was able to to understand

by

itself okay uh now let

me let me show you another

example of for completely different use

case so here we imagine that we have a

collection of documents and we want to

be able to generate answers based on

these

documents so the standard approach for

this is called retrial augmented

generation and this is not uh based on

an

agent and I would will show you it works

how it works uh and then I will show you

an alternative so it works as follows so

when you receive a question you don't

directly send it to the llm like this

you first perform a semantic

search to retrieve the relevant

paragraphs in your collection of

documents and then once you have these

chunks these paragraphs from your

initial collection documents you add

these chunks along with the with the

question in a prompt that you sent to an

llm so basically your prompt is

something like given the following facts

and then you put the relevant

paragraphs answer the following question

and then you put your

question so it works to a certain extent

but there are two problems with this

approach the first one is that the user

question might not be the right query

for the semantic search

so imagine that you are in the context

of a conversation and there are already

sever messages and the last message of

the user can be something like tell me

more and in this case it won't be very

useful to do a semantic search based on

this sentence uh because of course you

need to take into account all the

previous context of the conversation

just tell me more is not very

meaningful and then depending on the

question you might need to perform

either zero one or maybe several

searches so for example that can be

basic questions like if the user asks

how are how are you you don't need to do

a semantic search to answer this

question there can be some multihub

questions so questions um for which

actually you need to perform several

queries sequentially so imagine imagine

this question what are the main actors

of the latest movie of Martin's sces

first maybe you need to find what movie

it is and then you can ask about the

actors so here you would need to ask

these questions

sequentially and sometimes

also um you can have in fact several

questions in just one question so in

this case for example what are the

populations of Belgium and the

Netherlands maybe you want to ask two

questions in parallel

and as you have probably guessed by now

a good

alternative uh for for this approach is

to use an agent to which uh you give the

conversation history as an input and

then you make available to this agent a

single tool which is which is a Ral tool

which converts a query into a collection

of chunks and then you let the agent

define whether to use the

documents or not and you let the agent

choose the search query it wants to

use so let me show you how it works in

that

I okay so here I have a a collection of

documents like this so there

are uh several documents that are all

related to biodiversity there is a

report like this and then there is a

Wikipedia page like this and then two

other Wikipedia pages but in other other

formats just just for the

fun okay and then here you do something

which is not necessarily very

important um to to explain detail but

basically what you do here that you do a

pre-processing of the documents to be

able to do some semantic search and then

once you have done

this

then you can use it in the context of

this web app so here for example if I

ask a question which is very specific to

the collection of

documents for

example what does

itbs

104

this here I can get an answer including

with

the with the relevant extracts of my

documents

but then if I say for example can

you provide more

details like

this yeah so here it understood that uh

given the context of the whole

conversation uh you should not do a

semantic search with can you provide

more details but most likely it did

semantic search based on ipbs so look at

tring the logs to see what were the

queries so the first query here was let

me check okay

so for the first question it used the

Search tool which is the only tool

available here and it read this query

and for the second

tool it rephrase the question of the

user and ask for this semantic search

and then it was able to find a relevant

answer and to provide provide the

corresponding

sources okay so that's um yeah so that's

another way to to use an

agent okay so all this looks uh very

good and hopefully you already have some

ideas about how you can use agents in

your

organization but we should also be

mindful that agents can make mistakes

and that these mistakes can be quite

consequential

because by Design we allow the agent to

perform some actions

autonomously so we should then be very

cautious and we should make significant

efforts to make sure that these agents

are as robust as

possible so here we want to avoid both

format errors and substantial

errors format errors can be the

situations where the llm deviates from

the expected structure you know with the

loop with the thoughts the actions the

observations and format errors can be

also situation in which the llm simply

invent a tool that does not

exist um and if you have these format

erors um it can prevent the agent from

giving an

answer and there can be also substantial

errors so here maybe the llm can

perfectly comply with the expected

format but the actions taken will not be

adequate given the

context so how can we minimize the risk

of Errors so first and it works for both

format errors and substantial errors

it's important to use models that are

powerful enough for example GPT 3.5 or

GPT

4 at the end of the day um

leveraging virus tools being able to

determine the right course of action

it's it's quite demanding from a

cognitive perspective so it makes sense

that uh the performance for these agents

is quite dependent on the on how

powerful the models

are so of course it's also important to

perform tests that are as comprehensive

and realistic as

possible and now if we focus on form

materials so we can use various

technical tricks to constrain the text

generation and I will say a bit more

about this in a

minute then when it comes to Mi

mitigating the risks of substantial

errors so um so it can involve for

example iterating on all the

texts uh defining the tools so you have

first the description that are in in the

prompts but you have also the results

return by the tools and the messages

which are sent to the LM so all these

texts are read by the llm so you might

need to tweet them to make sure that

they are well understood by the

llm okay you can also um restrict on the

Fly the tools that are provided to the

agent and for this you need to take into

account the context

and I will also say a little bit more

about this later and finally you can uh

either put a human in the loop and make

sure that there is someone to validate

the actions suggested by the

llm or if you have try to implement your

your agent and you have performed all

your tests and then you realize that

certain tools are not well understood by

by the agents then at the end of the day

you can see simply remove these tools if

they are too um too risky to

use okay so I'd like to show how we

can um use some high level Frameworks to

reduce the risk of format erors so one

of these Frameworks is called

lmql and it's a great tool to Define

constraints on the text generated by an

LNM so in a nutshell um without giving

you too many deta details when you use

lmql sorry you write queries that

specify what the prompt and the text

generated will look like so that's the

green and purple Parts

here and the purple Parts in Brackets

are actually the parts that are

generated by the

LM you can also select a model and this

is the yellow part and you can can

Define the constraints to apply to the

text generated by the llm and this is

what is in Orange on the

left and this is very useful because it

gives fine brain control over the text

generated by the

LM and in the case of

Agents uh it's very useful because you

can make sure that you don't deviate

from the expected formats so the query

that you see on the screen would be

enough to implement an agent and you can

see that it's at the same time very

concise and very easy to read and here

we have a guarantee that um the the text

generated by the llm will precisely

follow the structure that we

expect okay another cool trick is the

use of um recent feature um announced by

openai in June and it's called function

calling so it's specific to certain open

a models like GPT 3.5 turbo or

gp4 and the idea that when you send the

compleation query to open AI you can

Beyond providing the prompt you can also

optionally specify one of one of several

functions and for each function you

describe the purpose of the function in

natural language in text and the format

of its input

for example if you if you look at this

example we Define a function to get the

current weather and we specify that we

expect two

inputs um a

location and the measurement unit here

whether it's Celsius degrees or

Fahrenheit

degrees and if you have specify this

function then the open API can answer

with a call of this function

and in this case it will provide the

inputs for this function so that's what

you see on the right and it's very it's

clearly very useful for agents and with

this we are not 100% sure that the the

expected format will be complied with

but it's much more likely that if than

if you don't use this

feature okay so what I've shown you

before so using Frameworks like lmql or

a feature like function cing this is

helpful for format errors but now I can

tell you more about a way to reduce

substantial errors so the idea here is

to adapt on the Fly the set of tools

made available to an agent and to do so

we take advantage we we do this on the

basis of the context so for example if

you know that the request form comes

from a certain customer

instead of giving the agent a very

generic toolbox you can give the the

agent a restricted toolbox that can only

impact the the account of this specific

customer uh so maybe the agent will make

some mistakes so it's it's still

possible but we know for sure that at

least there won't be any side effects um

that will affect any other

customer another example would be to not

include tools corresponding to services

that are not available for certain

customer for example services to which

this customer is not

eligible so the idea that in general you

should not give the agent more

flexibility that it needs to process the

current

request okay so that's all I wanted to

say about making agents more

robust so before we move on to the Q&A

part of the webinar like to share with

you some key

messages so first the barrier to build

the first agent is quite low and there

are two main reasons for this so the

first one is that the overall mechanism

for an agent is generic and on top of

this it has already been implemented in

software libraries like L

chain and the second reason is that you

can use of the Shelf model like GPT 3.5

GPT 4 and you don't need any kind of

fine tuning you just need to define the

tools for your

agent and even this is not very

complicated so my invitation for you

would be to build an agent today or

tomorrow maybe and it's not unrealistic

to imagine that you can have a first

working prototype in literally a few

hours

having said that even if it's easy to

create a first prototype you shouldn't

underestimate the efforts to deploy your

agent in real

life you will need some time to test it

to improve it um yeah you need to to

make sign significant effort before you

are confident that you can actually put

it in front of real

users and to help you with your agent

use cases that I has shared several uh

resources online so there is a section

on agents in our llm guide book the demo

that I showed you is available online so

you can download it you can reuse it for

your own use cases and we also recently

recently published a blog post on how to

implement agents in that

t and finally I'd like to say that if

you're dataq customer and if you have a

very concrete use case in mind we might

be able to help so don't hesitate to

contact us you can can contact either

your customer access manager or you can

contact me with the email on the slide

and we'll be very happy to discuss uh

and see how we can

collaborate so thank you very much for

your attention and I welcome your

questions thank you Vivian for uh for

that presentation uh as always uh I

learned a lot listening to it hi

everyone my name is Kurt I'm a colleague

of Vivian and joy and I'll just be

reading off some of the questions for

for Vivian so Vivian currently the uh uh

the most highly upvoted uh question is

about the the packages that were being

used uh in some of the different uh code

that we saw specifically about Lang

chain was that in there somewhere and uh

but also more generally what were uh

were the packages that you were

using I think that L chain is a very

very good tool to to put type agents

because you you almost don't need to do

anything you can just Define your tools

and then you can use various um types of

Agents so I would say that's a very good

starting point then if you need to do

some adjustments sometimes it's a bit

difficult because you don't have all the

flexibility sometimes you need to look

look at the code of long chain so it

might be a bit difficult but at least

for your initial prototype L chain is a

very good option

cool very clear uh the next question uh

is about the react example that you gave

at the uh the beginning of the

presentation you mentioned that you are

stopping text generation at the uh uh

the observation so it doesn't start

generating that itself um how did you do

that yeah actually it's quite simple let

me just find the right slide which is

this one so actually in the for example

if you use the open API there is a

parameter when you send a text

completion query you can specify certain

stop wordss so here what you would do is

that you would say um stop

equal um observation column and then if

you do this then the when you send the

prompt uh the open a API will stop

generating text as soon as it has

generated observation

col okay very clear

um and then the uh the next question uh

that we have is about a multi-agent

framework is it possible and beneficial

um or is it better just to stick with

one uh one agent which has access to

chains yeah that's a very nice question

um actually you can imagine that

um um you can imagine that um you can

create an agent and then you can use it

as a tool by another agent so you can

imagine to have various agents which

have uh very some specific skills and

that are good at certain things and then

you can have an agent which is a kind of

orchestrator which can select which

agent to

call a bit like in a team of human

beings you know there is a that can be a

manager and that can be various

resources and each has um

specialty it can make sense in

particular

if you know that you have a limited size

for the context window for the prompt so

you it would be sometimes can be

difficult to include everything so it

can be difficult to imagine to have a a

generate agent which has access to a

very very wide library of tools so it

can be better to First detect what

problem we want to address and then to

send it to the right agents so it it can

make a lot of sense to indeed Define um

multi multiple spe specialized agents

okay yeah there are very interesting

papers about this about that okay that's

great um I'm sure our audience might be

interested in some of those papers so I

think we we'll see if we can get some

links for the uh for the followup um we

did get a couple more questions just now

which is great so uh to the audience

please do continue to uh to submit them

um one of the questions is about scaling

up these uh these deployments and are

there particular tools or Frameworks

that you would like to remend for

recommend for performance

monitoring

um yeah what what I can show you is um

actually in in the coup how we can do

this

so here we implement the

agent um in this demo it's here in this

part

part and what we do is that we use ml

flow to perform some experiment tracking

so we keep we keep track of everything

that happens every time there is a new

iteration of the of the loop every time

we use a tool we track everything and

then we can follow this in experiment in

the experiment tracking part of the co

so that's what we do

here so here we can have various

indicators like the number of tools that

are used for example for each

um um here for each query and then we

can also for a

certain uh trajectory of an agent we can

look at all the details like uh what

were the value steps and how the Valu

tools were used so that's something that

that IQ users can use out of out of the

box so it's already something that is

useful

um and then I'm actually I'm not sure if

there are some specific

um packages that are useful for

monitoring agents it's possible it

change it so much but uh I would say

it's a very good starting point to use

experiment tracking or if you if you if

if you're not a data user you can use ml

flow and now there are certain there are

new features of ml flow that are

dedicated to to to

LMS very clear okay um the next question

that we have is about the uh retrieval

uh augmented Generation Um and

specifically what's the level of effort

required to prepare your own documents

to be used uh so specifically um how

hard is it to do that Vector

vectorization step um and uh I guess you

may want to add in there maybe there's

pre-processing as well OCR uh which may

be required even before vectorization

yeah so so actually it's um it's quite

simple today because there are very good

tools available so for example you can

use tools like long chain and other

tools or L index or

unstructured to uh to perform the

valious steps to prepare to do all this

preprocessing so there are VAR steps

first you need to extract the text from

the

documents then you need to split the

documents in small

paragraphs then you need to vectorize

this chunks so you need to find a vector

representation of these uh chunks that

correspond to their

meanings and then you need to index it

so that you can easily do a sematic

search so what you can see on the screen

here is two ways to do this in

the um so for example here we just use

long

chain to do everything to do

the uh the vectorization the the

indexing so since we use long chain it's

actually quite simple so yeah it's a few

dozen lines of code

and it could be even simpler because

here for example we try to make sure

that if we have

already done uh the indexing from some

documents and if we want to do it with

an updating of collection of documents

we don't do twice the indexing for the

document that we have already indexed

like something like this but if we so it

can be even simpler than this so it's

quite straightforward to do

it and if you use that I it's simpler

because you don't even need code you

just need what we call Visual

recipes um which don't need a single

line of code so here for example you

just do the text extraction from the Val

documents uh then you get

here um a table with the with the

content of the document and then based

on this you

can directly do the the splitting and

the vectorization in just one step just

with a few parameters that you need to

specify and then you can you can use

this to do your viral search so actually

this is very nowadays it's very simple

to do this um pre-processing Step for

rment in

generation okay very clear um next the

next question that we have up is uh

given all these examples that we've seen

uh do we have any guides that uh people

can follow along to implement these

examples themselves and I know we're in

the llm starter kit yeah yeah sure so

this uh this project it's a public demo

of um of using that LMS with

theu and it's pretty extensively

documented so you have a lot of

explanations for the various parts of of

the

flow and you can download the the

project so you need to to use the for

base uh and then you can import it on

your instance you you have here

the the specifications of the code

environments Etc so you have all the

technical prequisites and based on this

you should be able to to to use the demo

I showed you um for your information

also and it's um what we have

here so we also have this guide

book uh that presents all the Notions

interesting to to understand the

project and yeah based on this you

should be able

to yeah to implement these use cases

very easily so basically here if you

want to do the same but with your own uh

with your own tools you would just need

to to go to the library and just to to

replace these tools by your own

tools and then in the code which is

here uh you just need to replace these

tools by Yeah by your

own here you replace these tools by your

new tools and it will work if you don't

use data it's fine because with the link

that was in the latest slide

um show

you so here you will end up here and

then you will be able to to look at the

python

recipes and you will be able to recycle

maybe 90% of the of the Python code so

even if you don't use that I you will be

able to to to start from there that's

great thank you uh so we received a

question in the uh in the chat um and so

I think it's referring back to the um uh

the customer support use case uh that

you showed so having done you know that

uh that automated generation of

responses I think the question is

actually about can we do basic analytics

then on the um on the nature of

questions we've received and so on um

and I think that's probably one of the

the advantages of using a platform like

uh like da I is probably can uh do you

want to do you want to talk about that

with you yeah sure so of course you can

log all the requests and then I would

say it's a different use case it's a use

case of being able to classify the

questions in various categories either

categories that you have predetermined

or categories that you want to generate

on on the Fly and it's something yeah so

you can use

um either um you can find un model for

this categorization or you can just use

an llm for this as well and actually

that's more or less what we do in other

parts of this um sample

project so for example

here that's exactly what we do so we

have product review so it's it's

different so here we are talking about

product reviews but you can imagine that

this could be your uh customer

requests and here we have product

categories and we want to be able to

classify

this um these product reviews and these

categories so so you could do the same

with your customer requests and then

with the various categories that you

have defined and then you can just use

an llm it will be quite straightforward

you could just Define here the LM that

you want you can Define the vales

classes you can provide some examples uh

and it would be better in terms of

performance and then if you do this then

it will

be easy to get a a data set like this

with your predictions and even maybe

some explanations about your predictions

so yeah it's it's quite straightforward

to to do this very cool um we got a qu

another question in the chat uh this one

is uh more uh about recent developments

and specifically open AI uh Dev day um

and whether their announcements

regarding agents and functions uh will

change how this might be implemented in

the future um I don't know Vivian if

you've had a chance to look at that in

enough detail but if you have um I think

the audience would would be interested

on on your take um how will that uh you

know is open AI replacing Lang chain in

some way with uh with some of these

capabilities um yeah so it doesn't

change a lot compared to what has

already been announced in June with this

new function calling feature so I would

say that the specific feature that was

um announced on Monday was um um an

incremental Improvement so the idea is

that instead of giving the possibility

to the model to call one function it can

call um several functions at the same

time so for example if you have a

question so in the example that I showed

you we have something like

um something to get the weather and you

can imagine something else for example I

don't know get get the time for example

and if you ask in in a single question

if you ask for the time and the weather

at the same time now it's possible to do

both action at the same time so I'm not

sure it Chang changing a lot in any case

I think that it will still be useful to

have um L chain as a wrapper around this

because any

case the function calling feature of

open AI just is responsible all just to

call one tool or just one iteration so

any case you still need all the

orchestration all the react Loop you

still need this and it's not provided

in with with

open well very clear uh thank you for

the uh uh for the hot take on uh open a

latest uh uh developments so that's it

uh that's everything that we have for uh

for the questions um so I think at this

point uh uh we can thank you uh Vivian

for for the presentation um and of

course thank you to uh to all the

attendees for uh for tuning in um as Joy

said at the uh the outset we'll be uh

sharing the content uh and uh some of

the links uh afterward um and thank you

of course to everyone who uh who asked a

question these were uh these were great

questions so thanks to uh thanks to

everyone and just for your information I

put at the at the bottom of the de of

the slide I put the link to the deck so

you can just copy paste it if you want

to retrieve the the slides okay perfect

that's great thank you so much for uh

for that Vivian and once again thank you

for uh for the presentation thank you

very much bye

everyone