From Talk to Action: How LLMs Can Act in the Real World
Summary
TLDRВ этом видео представлены концепция и использование автономных агентов на основе больших языковых моделей (LLMs). Автор проиллюстрировал, как агенты могут преодолевать ограничения LLMs и применяться для решения реальных задач, таких как обработка запросов клиентов. Были представлены примеры и демонстрации, включая использование Python и инструментов машинного обучения для создания и улучшения стабильности агентов. Видео также обсуждает важность тестирования и адаптации агентов для их успешного развертывания в реальном мире.
Takeaways
- 🤖 Автономные агенты - это фундаментальный аспект больших языковых моделей, которые могут взаимодействовать с реальным миром.
- 📈 LLMs (большие языковые модели) удивительны тем, что они не только генерируют текст, но и выполняют широкий спектр задач обработки текста.
- 🚀 Агенты могут использовать внешние инструменты, что позволяет им преодолеть ограничения, с которыми сталкиваются LLMs.
- 🔧 Метод 'react' - это конкретный пример реализации агента, который использует структурированный подход для выполнения задач.
- 🔄 Процесс работы агента включает в себя цикл из трех частей: анализ текущей ситуации, выбор инструмента и наблюдение за результатами использования инструмента.
- 🛠️ Создание агента требует определения инструментов и написание Python-функций для выполнения различных действий.
- 📚 Примеры использования агентов включают обслуживание клиентов интернет-провайдера и генерацию ответов на основе документации.
- 🔒 Для обеспечения надежности агентов необходимо внимательно тестировать и улучшать их, а также ограничивать доступ к инструментам в соответствии с контекстом.
- 🔄 Использование инструментов, таких как LMQL и функционирование OpenAI, помогает снизить риски форматных и существенных ошибок у агентов.
- 🔄 Многоагентный подход может быть полезным для решения сложных задач, где различные агенты могут иметь специальные навыки и использоваться другими агентами для выполнения задач.
Q & A
Какие возможности автономных агентов в контексте больших языковых моделей?
-Автономные агенты позволяют большим языковым моделям (LLMs) преодолевать свои ограничения, такие как невозможность взаимодействия с реальным миром. Агенты могут использовать инструменты и принимать решения, основываясь на текстовых запросах, что позволяет им выполнять действия от имени пользователя.
Какие примеры использования автономных агентов были приведены в презентации?
-Два примера использования автономных агентов: обработка запросов клиентов интернет-провайдера (например, сброс пароля, диагностика интернет-соединения) и создание ответов на основе набора документов (например, генерация ответов по вопросам на основе текста документации).
Какие ограничения имеют большие языковые модели (LLMs)?
-Ограничения LLMs включают невозможность знания о событиях после определенной даты, трудности с выполнением определенных задач (например, математических операций) и то, что они могут возвращать только текстовые ответы без воздействия на реальный мир.
Какие основные компоненты включает в себя метод реализации агента, описанный в презентации?
-Метод реализации агента включает в себя написание промпта с тремя элементами: запросом пользователя, описанием доступных инструментов и структурой текста для генерации. Агент работает в цикле, анализируя ситуацию, выбирая инструмент и параметры, а затем做着 observation.
Какие инструменты можно использовать для увеличения надежности автономных агентов?
-Для увеличения надежности автономных агентов можно использовать мощные модели (например, GPT-3.5 или GPT-4), проводить комплексные и реалистичные тесты, использовать фреймворки для ограничения текстовой генерации (например, LMQL) и функциональность function calling, а также адаптировать набор доступных инструментов в соответствии с контекстом.
Какие функции были объявлены в рамках презентации для улучшения взаимодействия с автономными агентами?
-В рамках презентации были объявлены функции для получения текущей погоды и времени. Это было продемонстрировано с помощью функции function calling, которая позволяет модели выполнять несколько функций одновременно в ответ на запрос.
Какие ресурсы были предложены для помощи в реализации агентов на основе LLMs?
-Были предложены ресурсы, такие как LLM starter kit, демонстрирующий использование LMS с примерами и подробной документацией, а также блог-пост о реализации агентов. Для клиентов Datadog предоставляется возможность получения помощи от менеджеров или автора презентации.
Какие преимущества имеет использование Python-библиотек, таких как LChain, для реализации агентов?
-Python-библиотеки, такие как LChain, обеспечивают гибкость и простоту реализации агентов. Они позволяют определить инструменты и использовать общие шаблоны для прерывания текстовой генерации и обработки запросов без необходимости написания сложного кода.
Какие аспекты проекта можно повторно использовать для собственных случаев использования?
-Большая часть Python-кода из проекта можно повторно использовать, заменив инструменты на собственные. Также можно использовать визуальные рецепты для выполнения расчетов без написания кода.
Как обрабатывать многоэтапные или сложные запросы с помощью автономных агентов?
-Для обработки многоэтапных или сложных запросов автономные агенты могут использовать серию инструментов и определить последовательность действий на основе контекста и предыдущих действий. Это может включать отмену и перепланирование мероприятий или выполнение последовательности вопросов и ответов.
Какие инструменты можно использовать для мониторинга и анализа работы автономных агентов?
-Для мониторинга и анализа работы автономных агентов можно использовать инструменты, такие как MLFlow, для отслеживания экспериментов и показателей эффективности, а также для визуализации траектории агента и оценки используемых инструментов.
Outlines
🤖 Введение в генеративные AI и автономные агенты
В этом разделе представлен обзор генеративных AI и автономных агентов, а также их применение в реальном мире. Рассматриваются возможности и ограничения больших языковых моделей (LLMs), а также способы преодоления этих ограничений с помощью агентов. Основная цель презентации - убедить аудиторию в важности и эффективности автономных агентов.
🛠️ Как автономные агенты преодолевают ограничения LLMs
Раздел описывает, как автономные агенты могут использовать инструменты для преодоления ограничений LLMs. Приводятся примеры использования агентов в службе поддержки клиентов интернет-провайдера, а также метод реакции (react) для обработки запросов и выполнения действий. Основное внимание уделяется структуре текста, генерируемого агентом, и способам обработки результатов действий.
🌐 Примеры использования агентов в веб-приложениях
В этом разделе представлены примеры использования агентов в веб-приложениях для решения различных задач. Описывается создание фиктивного веб-приложения на основе питон-функций, которые отвечают за различные инструменты, и демонстрация работы агента в контексте этих инструментов. Рассматриваются также другие примеры использования агентов, включая генерацию ответов на основе документов и использование семантических поисков.
🔧 Улучшение надежности и безопасности автономных агентов
Раздел посвящен методам улучшения надежности и безопасности автономных агентов. Обсуждаются проблемы форматирования и существенных ошибок, а также способы их предотвращения. Рассматриваются инструменты и подходы, такие как использование мощных моделей, тестирование, определение ограничений для текста, адаптация набора инструментов в соответствии с контекстом и включение человека в цикл для проверки действий агента.
📋 Ресурсы и рекомендации для создания автономных агентов
В заключительном разделе предлагаются ключевые сообщения и рекомендации по созданию автономных агентов. Подчеркивается низкий порог входа для создания первого агента благодаря общедоступным инструментам и библиотекам, таким как L chain. Представлены ресурсы, такие как демонстрационный проект, руководство по агентам и блоговые статьи, а также контактная информация для получения помощи от команды разработчиков.
💬 Ответы на вопросы и обсуждение возможностей
В этом разделе авторы отвечают на вопросы аудитории, касающиеся использования пакетов и инструментов, остановки генерации текста, многоагентных фреймворков, мониторинга производительности, подготовки документов для ретривал генерации, а также возможностей использования платформы для анализа запросов и категоризации данных. Обсуждаются также последние разработки Open AI и их влияние на будущее реализации агентов.
Mindmap
Keywords
💡автономные агенты
💡генеративные AI
💡большие языковые модели (LLMs)
💡zero shot в контексте
💡few shot в контексте
💡реактивный метод
💡цикл обработки
💡функция вызова
💡LML
💡расширенная генерация
💡минимум
💡робастность
Highlights
Introduction to the concept of autonomous agents and their potential to overcome limitations of large language models (LLMs).
Explanation of why agents are needed and the limitations of LLMs that they can help address.
Demonstration of how agents work through two practical demos.
Discussion on making agents more robust, an essential consideration for real-life deployment.
The capability of LLMs in text generation and performing a wide range of text processing tasks.
The concept of zero-shot in-context learning and few-shot in-context learning.
Limitations of LLMs, such as ignorance of recent events, difficulty with certain tasks, and lack of real-world impact.
The idea of using tools to overcome LLM limitations, similar to how humans use tools to overcome their own limitations.
A concrete example of implementing an agent in a customer service scenario for an internet provider.
The method called 'react' for implementing an agent and its simple yet effective nature.
The importance of structuring the prompt correctly to achieve the desired outcome with an agent.
The potential for agents to be used in a variety of real-life use cases and their adaptability.
The use of frameworks like lmql and features like function calling to reduce the risk of format errors in agents.
Strategies for mitigating the risks of substantial errors in agent actions.
The low barrier to entry for building the first agent and the availability of resources for getting started.
The potential for multi-agent frameworks and the benefits of having specialized agents working in tandem.
The use of platforms like Datadog for performance monitoring and analytics of agent deployments.
The simplicity of preparing documents for retrieval-augmented generation with modern tools.
The availability of guides and resources for implementing the examples shown in the presentation.
The impact of recent developments from OpenAI on the implementation of agents and functions.
Transcripts
all right hi guys welcome today's to
today's generative AI boot camp session
from talk to action how large language
models can act in the real world we'll
give everyone a couple of seconds to
kind of trickle in um just a few
housekeeping notes uh you're welcome to
use the chat function but please leave
your questions in the question mark
section on the right at the end of the
today's session there will be a short
Q&A um Vivian I'm going to go ahead and
pass it over to
you thank you Joy and good morning good
afternoon everyone I'm Vian I'm director
of AI Consulting at theu and today I'm
delighted to talk to you
about let me just share my
screen okay it's
coming
yeah so I'm delighted to talk to you
today about the notion of autonomous
agents so in my opinion it's the most
fascinating topic when it comes to large
language models and my personal
objective for this presentation today is
to convince you or at least some of you
that it's indeed the
case so I will start by explaining why
we need agents in the first place and
what limitations of llms this agents can
help
overcome then I will show you how agents
actually work and I will illustrate this
with two
demos and I will finally talk about
making agents more robust uh which is an
essential consideration for deploying
these agents in real life and then we
can have a Q&A session based on the
question that you can start adding in
the Q&A
section okay let's start with the basics
so so we can probably all agree that
llms are very impressive so first they
are excellent at text generation so if
you provide a prompt like this one they
can suggest a coherent and credible text
that expands on this on this prompt so
it can look like this for example but so
far it's not really a surprise because
that's exactly what they have been
trained for but what is a bit more
intriguing is that you can do more than
Simple Text generation you can actually
perform a wide range of text processing
tasks and the trick here is to write a
prompt such that the expected answer is
exactly the most logical continuation of
the prompt so for example here imagine
that you want to perform a
text classification task uh and in which
you want to detect the sentiment of a
product review you can just describe the
task and the input text in the prompt
and then you are most likely to get a a
relevant answer so here we would call
this zero shot in context learning in
context because we add all the necessary
information in directly in the
prompt and you can also add a few
examples in the prompt and this helps
get better performance and in this case
we talk about few shot in context
learning so all this is very useful for
plenty of real life use
cases but at the same time llms also
suffer from some significant
limitations and I'm I'm sure that you
have already experienced some of these
limitations by interacting with
jgpt so first llms can know at most what
they have been shown during their
training so their training is mostly
based on internet
content um up to to certain date this
means that llms like the one powering
chpt for example are very likely to
ignore recent events or non-public
information then llms are quite bad at
certain tasks which don't seem very
difficult for us but for example if you
simply ask CH GPT to add two big numbers
it will probably fail and you can also
ask jgpt to for example provide 10 words
with exactly 15 letters and it's likely
to fail for at least some of these words
and finally and maybe it's the most
obvious
limitation you can send some text to an
llm and you can get some text in return
and this can be nice and can be useful
but it does not have any impact on the
real world so ideally would like AI
assistants not only to take part in
conversations but also to take actions
on our behalf in in the real
world so how can we overcome these
limitations so actually we can do
exactly as we do um we as human beings
when we Face our own
limitations and what we do we use tools
so for example if I don't have a very
good memory I can use a pen and a
scratch pad if I don't see very well I
can use a pair of glasses if I don't
know something I can use Wikipedia or I
can use an internet search engine and it
can be the same for llms we can give
them access to external tools and let
them decide whether and how they will
use these
tools and in this situation the llm is
referred to as an
agent so let me give you a more concrete
example so let's imagine that you're in
charge of the customer service of an
internet provider so you have plenty of
customers sending requests and questions
for example there is something wrong
with my internet connection can you
please help me or I forgot my password
or I'd like to sign up to a certain
option so you have all these requests
from customers and at the same time you
know technically how to reset a password
how to activate an option for customer
how to perform some diagnostics for an
internet connection
problem so you have all these tools at
your disposal but here the challenge is
not to actually perform this VI actions
but to be able to properly interpret the
requests of the customers which are
expressed in natural language and
English and you also need to plan the
right sequence of
actions so how can we do this um the
answer is that it's actually quite
simple um I'm I'm going to show you now
one of the methods to implement an agent
and this method is called
react let's see how it works so imagine
that we want to handle the the request
of a customer so for example here a
customer sends us a message uh I have a
problem with my internet connection can
you please solve this what we do is that
we first write a prompt which includes
three
elements so of course we'll include the
request of the customer a and
blue but we also include a description
of the various tools that the llm can
use and we will instruct the llm to
follow certain structure for the text it
will generate so the structure is the
loop with three parts at each iteration
of the loop the first part is a thought
which is an analysis of the current
situation the second part is the choice
of a tool and the choice of the
parameters to use the tool and the last
part is an observation which is the
result of the use of the
tool so we write this prompt and then we
ask the lln the only thing that it can
really do which is generating the
continuation a completion of the
promp so that's what we do and if we
have done our job well and if our prompt
is Well
written then the text generated will
follow the prescribed format so we'll
have first a
thought then we'll have an action
requested by the
llm so here it would be uh run
Diagnostics um and then the llm will
generate the word observation and here
we know that we need to
stop um because otherwise the LM will
keep going and it will simply
hallucinate the result of the action of
course the llm cannot know the outcome
of this action because it's not
connected to our it
system so it cannot know what would be
the result here of certain diagnos of
certain
Diagnostics so we stop the text
generation and now we can pass the
purple
segment um and we can identify the
action that was requested so here when
Diagnostics we can identify the input
parameters of this action so here the
input parameter would be the ID of the
customer which is
1223 and then we can perform this action
independently from the llm using our own
internal API of our of our it
system and once we get the result of
this action like this we can append it
to all the text generated so far and we
can send all this the green part the
blue part the purple parts we can send
this again to the
llm and then the llm will just keep
going so again it will generate a
thought an action it will write
observation and here again we stop the
text generation we extract the requested
action we identify the parameter of this
action we per from the action outside of
the llm we get a result and then we
append it to all the texts generated so
far and we do this again and again until
we get an answer for our question and
that's it so you see it's quite
simple and the crazy thing that just
doing this works pretty
well and here there
are when you look at this method there
are important things to note so the
first one is that all the details
specific to your use
case um in particular everything related
to the tools that you make available to
the agent all these details are included
in the prompt they are not in the model
they are in the prompt this means this
mean that you don't need an llm that was
specifically trained to implement this
method or to implement your specific
toolbox
you just need a model that is powerful
enough to accomplish this task so you
can use a another the Shelf model you
don't need to train or to fune
anything and then um what is also
interesting that the method is quite
generic so you see that all the logic
the template for your prompt um the way
you interupt the text generation the way
you part the message all this does not
depend on your um on your tools on your
specific on the specific details of your
use case so it mean that you can reuse
all this logic from one use case to
another you don't need to develop it
again every time and actually there are
some implementation that you can
directly use in popular python libraries
like Lun chain so you don't even need to
code this so at the end of the day the
only thing that you need to specify for
your uh on your case on your case is the
definition of the tools and now we're
going to to see how it works so how do
we specify these tools so we need three
things first we need a name and a
description to explain in the initial
prompt what tools are available to the
LM the name is also used during the the
loop
uh to identify the actions that are
requested by the
LM and finally you need some code to
perform the corresponding action when
requested by the
LM but in fact if you are able to write
a python function to perform this action
in fact you already have a name a
description and the right code so for
the name you can just reuse the name of
the
function um for the descript ion you can
take the documentation of your function
and um of course you can directly and
the function itself can be the code that
is triggered by the
llm so at the end of the day the only
thing you need to implement an agent is
to be able to write python functions for
all the actions that need to be
performed by the llm so it's not
difficult at
all okay let's now see an agent in
action with
demo uh so I
created um let me show you I created in
datu several python functions that
correspond to uh the various tools of
the internet provider in this example so
all this is a fictitious use case but
basically here you can see that I
defined some functions so there is one
to reset pass the password of the
customer want to schedule the visit by
technician you can
run some Diagnostics on an internet
connection you can sign up to an option
Etc um and now I can show you a web app
that implements an agent on the basis of
these tools so it looks like this on the
left you can see the code on the right
you can see the resulting web app and um
I won't go I won't go through the code
in detail but the only thing that you
need to remember that it's generic so
here this is not specific to this to the
particular collection of tools that I
make available to the agent
here um so if you had to to change the
tools you could just reuse the same the
same
code so now here let me try an examples
I will say that here I have a problem
with my internet
connection do something
see the results that we
get okay should take a few seconds yeah
so here you can see that the the agent
uh chose a few actions so first it
decided to run some Diagnostics on the
internet connection it got the answer
here and based on this outcome of the
first action it decided as the second
action to organize the visit by
technician and then it understood that
it was enough that there were there were
no actions anymore that were necessary
and then it drafted the following
message let me show you another example
so now imagine that we have organized
this visits with the technician and
now um for example we can can say okay
I'm not
available anym
uh for the
visit by technician on
Monday and we please rky
go okay let's see how it
goes so again we see that the sequence
of actions that were determined by the
agent is quite logical so first it
cancels the appoint
M uh it gets the confirmation of the
consolation and then it knows that it it
needs to schedule a new
appointment and that's what it does and
then it know that it's uh that it's over
and it can draft this answer which is
relevant given the
context so in in this example it's
interesting
because in uh in the way we Define the
function we don't have a function to
reschedule an existing meeting we have
just given a series of tools and two of
these tools are uh how to cancel an
appointment and how to plan another
appointment but we never had explicitly
to explain to the agent that
rescheduling corresponds to cancelling
and to plan a new meeting uh that's what
the the model was able to to understand
by
itself okay uh now let
me let me show you another
example of for completely different use
case so here we imagine that we have a
collection of documents and we want to
be able to generate answers based on
these
documents so the standard approach for
this is called retrial augmented
generation and this is not uh based on
an
agent and I would will show you it works
how it works uh and then I will show you
an alternative so it works as follows so
when you receive a question you don't
directly send it to the llm like this
you first perform a semantic
search to retrieve the relevant
paragraphs in your collection of
documents and then once you have these
chunks these paragraphs from your
initial collection documents you add
these chunks along with the with the
question in a prompt that you sent to an
llm so basically your prompt is
something like given the following facts
and then you put the relevant
paragraphs answer the following question
and then you put your
question so it works to a certain extent
but there are two problems with this
approach the first one is that the user
question might not be the right query
for the semantic search
so imagine that you are in the context
of a conversation and there are already
sever messages and the last message of
the user can be something like tell me
more and in this case it won't be very
useful to do a semantic search based on
this sentence uh because of course you
need to take into account all the
previous context of the conversation
just tell me more is not very
meaningful and then depending on the
question you might need to perform
either zero one or maybe several
searches so for example that can be
basic questions like if the user asks
how are how are you you don't need to do
a semantic search to answer this
question there can be some multihub
questions so questions um for which
actually you need to perform several
queries sequentially so imagine imagine
this question what are the main actors
of the latest movie of Martin's sces
first maybe you need to find what movie
it is and then you can ask about the
actors so here you would need to ask
these questions
sequentially and sometimes
also um you can have in fact several
questions in just one question so in
this case for example what are the
populations of Belgium and the
Netherlands maybe you want to ask two
questions in parallel
and as you have probably guessed by now
a good
alternative uh for for this approach is
to use an agent to which uh you give the
conversation history as an input and
then you make available to this agent a
single tool which is which is a Ral tool
which converts a query into a collection
of chunks and then you let the agent
define whether to use the
documents or not and you let the agent
choose the search query it wants to
use so let me show you how it works in
that
I okay so here I have a a collection of
documents like this so there
are uh several documents that are all
related to biodiversity there is a
report like this and then there is a
Wikipedia page like this and then two
other Wikipedia pages but in other other
formats just just for the
fun okay and then here you do something
which is not necessarily very
important um to to explain detail but
basically what you do here that you do a
pre-processing of the documents to be
able to do some semantic search and then
once you have done
this
then you can use it in the context of
this web app so here for example if I
ask a question which is very specific to
the collection of
documents for
example what does
itbs
104
this here I can get an answer including
with
the with the relevant extracts of my
documents
but then if I say for example can
you provide more
details like
this yeah so here it understood that uh
given the context of the whole
conversation uh you should not do a
semantic search with can you provide
more details but most likely it did
semantic search based on ipbs so look at
tring the logs to see what were the
queries so the first query here was let
me check okay
so for the first question it used the
Search tool which is the only tool
available here and it read this query
and for the second
tool it rephrase the question of the
user and ask for this semantic search
and then it was able to find a relevant
answer and to provide provide the
corresponding
sources okay so that's um yeah so that's
another way to to use an
agent okay so all this looks uh very
good and hopefully you already have some
ideas about how you can use agents in
your
organization but we should also be
mindful that agents can make mistakes
and that these mistakes can be quite
consequential
because by Design we allow the agent to
perform some actions
autonomously so we should then be very
cautious and we should make significant
efforts to make sure that these agents
are as robust as
possible so here we want to avoid both
format errors and substantial
errors format errors can be the
situations where the llm deviates from
the expected structure you know with the
loop with the thoughts the actions the
observations and format errors can be
also situation in which the llm simply
invent a tool that does not
exist um and if you have these format
erors um it can prevent the agent from
giving an
answer and there can be also substantial
errors so here maybe the llm can
perfectly comply with the expected
format but the actions taken will not be
adequate given the
context so how can we minimize the risk
of Errors so first and it works for both
format errors and substantial errors
it's important to use models that are
powerful enough for example GPT 3.5 or
GPT
4 at the end of the day um
leveraging virus tools being able to
determine the right course of action
it's it's quite demanding from a
cognitive perspective so it makes sense
that uh the performance for these agents
is quite dependent on the on how
powerful the models
are so of course it's also important to
perform tests that are as comprehensive
and realistic as
possible and now if we focus on form
materials so we can use various
technical tricks to constrain the text
generation and I will say a bit more
about this in a
minute then when it comes to Mi
mitigating the risks of substantial
errors so um so it can involve for
example iterating on all the
texts uh defining the tools so you have
first the description that are in in the
prompts but you have also the results
return by the tools and the messages
which are sent to the LM so all these
texts are read by the llm so you might
need to tweet them to make sure that
they are well understood by the
llm okay you can also um restrict on the
Fly the tools that are provided to the
agent and for this you need to take into
account the context
and I will also say a little bit more
about this later and finally you can uh
either put a human in the loop and make
sure that there is someone to validate
the actions suggested by the
llm or if you have try to implement your
your agent and you have performed all
your tests and then you realize that
certain tools are not well understood by
by the agents then at the end of the day
you can see simply remove these tools if
they are too um too risky to
use okay so I'd like to show how we
can um use some high level Frameworks to
reduce the risk of format erors so one
of these Frameworks is called
lmql and it's a great tool to Define
constraints on the text generated by an
LNM so in a nutshell um without giving
you too many deta details when you use
lmql sorry you write queries that
specify what the prompt and the text
generated will look like so that's the
green and purple Parts
here and the purple Parts in Brackets
are actually the parts that are
generated by the
LM you can also select a model and this
is the yellow part and you can can
Define the constraints to apply to the
text generated by the llm and this is
what is in Orange on the
left and this is very useful because it
gives fine brain control over the text
generated by the
LM and in the case of
Agents uh it's very useful because you
can make sure that you don't deviate
from the expected formats so the query
that you see on the screen would be
enough to implement an agent and you can
see that it's at the same time very
concise and very easy to read and here
we have a guarantee that um the the text
generated by the llm will precisely
follow the structure that we
expect okay another cool trick is the
use of um recent feature um announced by
openai in June and it's called function
calling so it's specific to certain open
a models like GPT 3.5 turbo or
gp4 and the idea that when you send the
compleation query to open AI you can
Beyond providing the prompt you can also
optionally specify one of one of several
functions and for each function you
describe the purpose of the function in
natural language in text and the format
of its input
for example if you if you look at this
example we Define a function to get the
current weather and we specify that we
expect two
inputs um a
location and the measurement unit here
whether it's Celsius degrees or
Fahrenheit
degrees and if you have specify this
function then the open API can answer
with a call of this function
and in this case it will provide the
inputs for this function so that's what
you see on the right and it's very it's
clearly very useful for agents and with
this we are not 100% sure that the the
expected format will be complied with
but it's much more likely that if than
if you don't use this
feature okay so what I've shown you
before so using Frameworks like lmql or
a feature like function cing this is
helpful for format errors but now I can
tell you more about a way to reduce
substantial errors so the idea here is
to adapt on the Fly the set of tools
made available to an agent and to do so
we take advantage we we do this on the
basis of the context so for example if
you know that the request form comes
from a certain customer
instead of giving the agent a very
generic toolbox you can give the the
agent a restricted toolbox that can only
impact the the account of this specific
customer uh so maybe the agent will make
some mistakes so it's it's still
possible but we know for sure that at
least there won't be any side effects um
that will affect any other
customer another example would be to not
include tools corresponding to services
that are not available for certain
customer for example services to which
this customer is not
eligible so the idea that in general you
should not give the agent more
flexibility that it needs to process the
current
request okay so that's all I wanted to
say about making agents more
robust so before we move on to the Q&A
part of the webinar like to share with
you some key
messages so first the barrier to build
the first agent is quite low and there
are two main reasons for this so the
first one is that the overall mechanism
for an agent is generic and on top of
this it has already been implemented in
software libraries like L
chain and the second reason is that you
can use of the Shelf model like GPT 3.5
GPT 4 and you don't need any kind of
fine tuning you just need to define the
tools for your
agent and even this is not very
complicated so my invitation for you
would be to build an agent today or
tomorrow maybe and it's not unrealistic
to imagine that you can have a first
working prototype in literally a few
hours
having said that even if it's easy to
create a first prototype you shouldn't
underestimate the efforts to deploy your
agent in real
life you will need some time to test it
to improve it um yeah you need to to
make sign significant effort before you
are confident that you can actually put
it in front of real
users and to help you with your agent
use cases that I has shared several uh
resources online so there is a section
on agents in our llm guide book the demo
that I showed you is available online so
you can download it you can reuse it for
your own use cases and we also recently
recently published a blog post on how to
implement agents in that
t and finally I'd like to say that if
you're dataq customer and if you have a
very concrete use case in mind we might
be able to help so don't hesitate to
contact us you can can contact either
your customer access manager or you can
contact me with the email on the slide
and we'll be very happy to discuss uh
and see how we can
collaborate so thank you very much for
your attention and I welcome your
questions thank you Vivian for uh for
that presentation uh as always uh I
learned a lot listening to it hi
everyone my name is Kurt I'm a colleague
of Vivian and joy and I'll just be
reading off some of the questions for
for Vivian so Vivian currently the uh uh
the most highly upvoted uh question is
about the the packages that were being
used uh in some of the different uh code
that we saw specifically about Lang
chain was that in there somewhere and uh
but also more generally what were uh
were the packages that you were
using I think that L chain is a very
very good tool to to put type agents
because you you almost don't need to do
anything you can just Define your tools
and then you can use various um types of
Agents so I would say that's a very good
starting point then if you need to do
some adjustments sometimes it's a bit
difficult because you don't have all the
flexibility sometimes you need to look
look at the code of long chain so it
might be a bit difficult but at least
for your initial prototype L chain is a
very good option
cool very clear uh the next question uh
is about the react example that you gave
at the uh the beginning of the
presentation you mentioned that you are
stopping text generation at the uh uh
the observation so it doesn't start
generating that itself um how did you do
that yeah actually it's quite simple let
me just find the right slide which is
this one so actually in the for example
if you use the open API there is a
parameter when you send a text
completion query you can specify certain
stop wordss so here what you would do is
that you would say um stop
equal um observation column and then if
you do this then the when you send the
prompt uh the open a API will stop
generating text as soon as it has
generated observation
col okay very clear
um and then the uh the next question uh
that we have is about a multi-agent
framework is it possible and beneficial
um or is it better just to stick with
one uh one agent which has access to
chains yeah that's a very nice question
um actually you can imagine that
um um you can imagine that um you can
create an agent and then you can use it
as a tool by another agent so you can
imagine to have various agents which
have uh very some specific skills and
that are good at certain things and then
you can have an agent which is a kind of
orchestrator which can select which
agent to
call a bit like in a team of human
beings you know there is a that can be a
manager and that can be various
resources and each has um
specialty it can make sense in
particular
if you know that you have a limited size
for the context window for the prompt so
you it would be sometimes can be
difficult to include everything so it
can be difficult to imagine to have a a
generate agent which has access to a
very very wide library of tools so it
can be better to First detect what
problem we want to address and then to
send it to the right agents so it it can
make a lot of sense to indeed Define um
multi multiple spe specialized agents
okay yeah there are very interesting
papers about this about that okay that's
great um I'm sure our audience might be
interested in some of those papers so I
think we we'll see if we can get some
links for the uh for the followup um we
did get a couple more questions just now
which is great so uh to the audience
please do continue to uh to submit them
um one of the questions is about scaling
up these uh these deployments and are
there particular tools or Frameworks
that you would like to remend for
recommend for performance
monitoring
um yeah what what I can show you is um
actually in in the coup how we can do
this
so here we implement the
agent um in this demo it's here in this
part
part and what we do is that we use ml
flow to perform some experiment tracking
so we keep we keep track of everything
that happens every time there is a new
iteration of the of the loop every time
we use a tool we track everything and
then we can follow this in experiment in
the experiment tracking part of the co
so that's what we do
here so here we can have various
indicators like the number of tools that
are used for example for each
um um here for each query and then we
can also for a
certain uh trajectory of an agent we can
look at all the details like uh what
were the value steps and how the Valu
tools were used so that's something that
that IQ users can use out of out of the
box so it's already something that is
useful
um and then I'm actually I'm not sure if
there are some specific
um packages that are useful for
monitoring agents it's possible it
change it so much but uh I would say
it's a very good starting point to use
experiment tracking or if you if you if
if you're not a data user you can use ml
flow and now there are certain there are
new features of ml flow that are
dedicated to to to
LMS very clear okay um the next question
that we have is about the uh retrieval
uh augmented Generation Um and
specifically what's the level of effort
required to prepare your own documents
to be used uh so specifically um how
hard is it to do that Vector
vectorization step um and uh I guess you
may want to add in there maybe there's
pre-processing as well OCR uh which may
be required even before vectorization
yeah so so actually it's um it's quite
simple today because there are very good
tools available so for example you can
use tools like long chain and other
tools or L index or
unstructured to uh to perform the
valious steps to prepare to do all this
preprocessing so there are VAR steps
first you need to extract the text from
the
documents then you need to split the
documents in small
paragraphs then you need to vectorize
this chunks so you need to find a vector
representation of these uh chunks that
correspond to their
meanings and then you need to index it
so that you can easily do a sematic
search so what you can see on the screen
here is two ways to do this in
the um so for example here we just use
long
chain to do everything to do
the uh the vectorization the the
indexing so since we use long chain it's
actually quite simple so yeah it's a few
dozen lines of code
and it could be even simpler because
here for example we try to make sure
that if we have
already done uh the indexing from some
documents and if we want to do it with
an updating of collection of documents
we don't do twice the indexing for the
document that we have already indexed
like something like this but if we so it
can be even simpler than this so it's
quite straightforward to do
it and if you use that I it's simpler
because you don't even need code you
just need what we call Visual
recipes um which don't need a single
line of code so here for example you
just do the text extraction from the Val
documents uh then you get
here um a table with the with the
content of the document and then based
on this you
can directly do the the splitting and
the vectorization in just one step just
with a few parameters that you need to
specify and then you can you can use
this to do your viral search so actually
this is very nowadays it's very simple
to do this um pre-processing Step for
rment in
generation okay very clear um next the
next question that we have up is uh
given all these examples that we've seen
uh do we have any guides that uh people
can follow along to implement these
examples themselves and I know we're in
the llm starter kit yeah yeah sure so
this uh this project it's a public demo
of um of using that LMS with
theu and it's pretty extensively
documented so you have a lot of
explanations for the various parts of of
the
flow and you can download the the
project so you need to to use the for
base uh and then you can import it on
your instance you you have here
the the specifications of the code
environments Etc so you have all the
technical prequisites and based on this
you should be able to to to use the demo
I showed you um for your information
also and it's um what we have
here so we also have this guide
book uh that presents all the Notions
interesting to to understand the
project and yeah based on this you
should be able
to yeah to implement these use cases
very easily so basically here if you
want to do the same but with your own uh
with your own tools you would just need
to to go to the library and just to to
replace these tools by your own
tools and then in the code which is
here uh you just need to replace these
tools by Yeah by your
own here you replace these tools by your
new tools and it will work if you don't
use data it's fine because with the link
that was in the latest slide
um show
you so here you will end up here and
then you will be able to to look at the
python
recipes and you will be able to recycle
maybe 90% of the of the Python code so
even if you don't use that I you will be
able to to to start from there that's
great thank you uh so we received a
question in the uh in the chat um and so
I think it's referring back to the um uh
the customer support use case uh that
you showed so having done you know that
uh that automated generation of
responses I think the question is
actually about can we do basic analytics
then on the um on the nature of
questions we've received and so on um
and I think that's probably one of the
the advantages of using a platform like
uh like da I is probably can uh do you
want to do you want to talk about that
with you yeah sure so of course you can
log all the requests and then I would
say it's a different use case it's a use
case of being able to classify the
questions in various categories either
categories that you have predetermined
or categories that you want to generate
on on the Fly and it's something yeah so
you can use
um either um you can find un model for
this categorization or you can just use
an llm for this as well and actually
that's more or less what we do in other
parts of this um sample
project so for example
here that's exactly what we do so we
have product review so it's it's
different so here we are talking about
product reviews but you can imagine that
this could be your uh customer
requests and here we have product
categories and we want to be able to
classify
this um these product reviews and these
categories so so you could do the same
with your customer requests and then
with the various categories that you
have defined and then you can just use
an llm it will be quite straightforward
you could just Define here the LM that
you want you can Define the vales
classes you can provide some examples uh
and it would be better in terms of
performance and then if you do this then
it will
be easy to get a a data set like this
with your predictions and even maybe
some explanations about your predictions
so yeah it's it's quite straightforward
to to do this very cool um we got a qu
another question in the chat uh this one
is uh more uh about recent developments
and specifically open AI uh Dev day um
and whether their announcements
regarding agents and functions uh will
change how this might be implemented in
the future um I don't know Vivian if
you've had a chance to look at that in
enough detail but if you have um I think
the audience would would be interested
on on your take um how will that uh you
know is open AI replacing Lang chain in
some way with uh with some of these
capabilities um yeah so it doesn't
change a lot compared to what has
already been announced in June with this
new function calling feature so I would
say that the specific feature that was
um announced on Monday was um um an
incremental Improvement so the idea is
that instead of giving the possibility
to the model to call one function it can
call um several functions at the same
time so for example if you have a
question so in the example that I showed
you we have something like
um something to get the weather and you
can imagine something else for example I
don't know get get the time for example
and if you ask in in a single question
if you ask for the time and the weather
at the same time now it's possible to do
both action at the same time so I'm not
sure it Chang changing a lot in any case
I think that it will still be useful to
have um L chain as a wrapper around this
because any
case the function calling feature of
open AI just is responsible all just to
call one tool or just one iteration so
any case you still need all the
orchestration all the react Loop you
still need this and it's not provided
in with with
open well very clear uh thank you for
the uh uh for the hot take on uh open a
latest uh uh developments so that's it
uh that's everything that we have for uh
for the questions um so I think at this
point uh uh we can thank you uh Vivian
for for the presentation um and of
course thank you to uh to all the
attendees for uh for tuning in um as Joy
said at the uh the outset we'll be uh
sharing the content uh and uh some of
the links uh afterward um and thank you
of course to everyone who uh who asked a
question these were uh these were great
questions so thanks to uh thanks to
everyone and just for your information I
put at the at the bottom of the de of
the slide I put the link to the deck so
you can just copy paste it if you want
to retrieve the the slides okay perfect
that's great thank you so much for uh
for that Vivian and once again thank you
for uh for the presentation thank you
very much bye
everyone
Посмотреть больше похожих видео
Обновленный Selenium и работа с прокси | Python, Selenium и proxy | Подмена IP адреса
Не размещайте резюме на hh.ru, пока не посмотрите это видео
Как ЗАРАБОТАТЬ на Нейросети в 2023? 5 Нейросетей и 5 способов - 300$ за 30 Минут
Rory Sutherland | Books that make you think differently
ИЗУЧИТЕ ГЛАВНУЮ НЕЙРОСЕТЬ МИРА от Б до Ю
1 Блок 7
5.0 / 5 (0 votes)