Big AutoGen UPDATE 0.2.28 | Databricks Integration 🎉

Summary

视频脚本介绍了Autogen版本0.228的更新，包括GPT助手代理、群聊恢复功能、文本压缩工具以及与Databricks的集成。GPT助手代理利用OpenAI的API，支持代码解释器、文件搜索等内置工具，并通过线程技术优化消息历史存储。群聊恢复功能允许用户通过传递先前对话的消息来继续之前的群聊。文本压缩工具LLM Lingua旨在提高LLM操作的效率和成本效益。Databricks集成展示了如何将Databricks的通用LLM与Autogen集成，为用户提供更多模型选择和集成可能性。这些更新为Autogen框架带来更广泛的应用场景和灵活性。

Takeaways

• 🆕 Autogen发布了0.228版本更新，带来了许多新功能和改进。
• 🤖 新增GPT助手代理，由OpenAI助手API支持，可以利用代码解释器、文件搜索和函数调用等内置工具。
• 📑 助手代理能够生成文件，如图像和电子表格，并通过线程自动存储消息历史并调整上下文长度。
• 🔄 引入了群聊恢复功能，允许通过传递先前对话的消息来恢复之前的群聊。
• 🔗 展示了如何通过代码示例继续已终止的对话，包括如何加载JSON字符串或消息列表。
• 🗜️ 介绍了使用LLM Lingua压缩文本的工具，有效提高了LLM操作的效率和成本效益。
• 📚 举例说明了如何使用LLM Lingua压缩Autogen研究论文，节省了近20,000个token。
• 🔧 提供了如何将文本压缩器与Autogen代理集成的示例，展示了如何在研究代理中使用文本压缩器。
• 👨‍💻 Autogen对.NET开发者的支持明显，有多个.NET相关的更新和示例。
• 🔗 Autogen正在增加与其他服务的集成，如Databricks，为用户提供了更多的模型选择和集成可能性。
• 📈 Databricks的dbrx是一个通用的LLM，为开放的LLM设定了新标准，并在Hugging Face上开源了模型。

Q & A

Autogen的最新更新版本是什么？

Autogen的最新更新版本是0.228。

GPT助理代理是什么，它有哪些功能？

GPT助理代理是由Open AI助理API支持的代理，可以使用多种工具，如代码解释器、文件搜索和功能调用等内置工具。此外，它还可以生成文件，如图像和电子表格，并且可以利用线程自动存储消息历史并根据模型的上下文长度进行调整。

如何设置GPT助理代理？

设置GPT助理代理非常简单，只需创建一个GPT助理代理，并定义助理配置，指定想要代理具备的工具或内置功能。

什么是恢复群聊功能，它如何工作？

恢复群聊功能允许用户通过传递先前对话的消息给群聊管理器的恢复函数来恢复之前的群聊。恢复函数会返回最后一条消息和最后一个代理，这些可以用来重新启动聊天。

如何使用文本压缩工具LLM Lingua？

LLM Lingua是一个设计用来压缩提示的工具，可以有效地提高LLM操作的效率和成本效益。通过实例化文本消息压缩器对象，并将其应用于提取的文本，可以节省大量令牌，从而节省成本并增加上下文窗口中的令牌数量。

如何在Autogen代理中集成LLM Lingua进行文本压缩？

在Autogen代理中集成LLM Lingua进行文本压缩，需要在代理设置中添加文本压缩处理，通过transform messages with the text compressor来实现。

Autogen是否支持.NET开发者？

是的，Autogen支持.NET开发者，并且在更新中包含了多个.NET相关的更新和示例。

Autogen有哪些集成示例？

Autogen提供了多种集成示例，包括与Databricks的集成、加密交易代理、虚拟焦点小组等。

Databricks的dbrx是什么，它与Autogen如何集成？

Databricks的dbrx是一个通用的LLM，为开放LLM设定了新的标准。它与Autogen的集成示例包括设置API令牌，并通过Databricks主机或ADS工作区进行配置，以实现基本的聊天功能。

如何通过Autogen框架使用不同的模型或服务？

Autogen框架通过集成不同的服务和模型，如Databricks的dbrx，为用户提供了使用不同模型或服务的可能性。用户可以根据个人喜好选择使用Open AI的Assistant API或尝试其他类型的模型。

Outlines

00:00

🔄 Autogen 0.228版本更新概览

本次更新介绍了Autogen 0.228版本，其中包括了多项新功能和改进。首先，新增了基于Open AI助手API的GPT助手代理，该代理支持代码解释器、文件搜索和函数调用等内置工具。此外，还引入了'threads'功能，它能够自动存储消息历史并根据模型的上下文长度进行调整。用户还可以利用助手生成图像和电子表格等文件。另一个亮点是群聊恢复功能，允许用户通过传递先前对话的消息来恢复之前的群聊。示例中展示了如何设置群聊对象，并使用'resume'功能来继续之前的对话。

05:02

📄 利用LLM Lingua压缩文本提高效率

介绍了使用LLM Lingua工具压缩提示的功能，这可以有效地提高大型语言模型（LLM）操作的效率和成本效益。示例中展示了如何将一篇研究论文的文本进行压缩，通过应用LLM Lingua的文本压缩器，节省了近20,000个token。这对于具有较小上下文窗口的模型尤其有用，因为它可以帮助用户在有限的上下文中保存更多的信息。此外，还展示了如何将文本压缩器集成到Autogen代理中，以自动处理文本压缩，从而在保持关键信息的同时节省成本。

10:02

🛠️ Autogen与Data Bricks的集成示例

讨论了Autogen与Data Bricks的集成，Data Bricks是一个提供通用大型语言模型（LLM）的平台，其模型可在Hugging Face上找到。文档中提供了如何将Data Bricks与Autogen集成的示例，包括设置API令牌和基本的'hello world'示例。此外，还提供了一个简单的编码代理示例，展示了如何使用Data Bricks助手代理进行基本操作。这表明Autogen正在扩展其集成能力，为用户提供更多的可能性和灵活性。

15:04

🎉 Autogen更新带来的新机遇

总结了Autogen更新带来的新机遇，强调了框架的开放性，允许用户尝试不同的模型和工具。提到了Autogen开始集成更多服务，如Data Bricks，这为用户提供了更多的选择和灵活性。还提到了Autogen对.NET开发者的支持，以及社区中分享的各种应用和集成示例，如加密交易代理和虚拟焦点小组。最后，鼓励用户尝试这些更新，并提供了一个Autogen初学者课程链接，帮助用户更好地理解Autogen。

Keywords

💡Autogen

Autogen是一个框架，它允许用户创建和运行各种智能代理。在视频中，Autogen是更新的核心主题，讨论了其最新版本0.228的多个功能和改进。例如，它提到了Autogen的GPT助手代理，这是通过Open AI助手API支持的，以及如何使用Autogen进行群聊恢复。

💡GPT助手代理

GPT助手代理是Autogen更新的一部分，它利用Open AI助手API提供多种工具，如代码解释器、文件搜索和函数调用。这些工具是OpenAI内置的，允许用户执行复杂任务，例如生成文件或恢复群聊。在视频中，提到了如何通过设置助手配置来定义这些工具。

💡代码解释器

代码解释器是Autogen更新中提到的一个工具，它允许用户执行和测试代码。在视频中，它被提及为GPT助手代理可以使用的工具之一，这表明用户可以利用这个功能来增强他们的自动化脚本或智能代理。

💡文件搜索

文件搜索是Autogen更新中引入的另一个工具，它使智能代理能够搜索和访问文件系统中的文件。视频中提到，这个功能可以集成到GPT助手代理中，从而提高代理处理和响应用户请求的能力。

💡函数调用

函数调用是Autogen更新中的一个特性，它允许用户定义特定的函数，并通过API模式在智能代理中使用这些函数。视频中给出了一个例子，如何定义一个获取当前天气的函数，并在助手配置中将其设置为代理的工具。

💡群聊恢复

群聊恢复是Autogen更新中的一个新功能，它允许用户恢复之前终止的群聊。视频中详细讨论了如何通过传递先前对话的消息给群聊管理器的恢复函数来实现这一点。这个功能可以让用户继续之前的讨论，而不需要重新开始。

💡LLM Lingua

LLM Lingua是视频中提到的一个文本压缩工具，它旨在提高大型语言模型（LLM）操作的效率和成本效益。通过压缩提示，它可以减少所需的令牌数量，从而节省成本并适应具有较小上下文窗口的模型。

💡文本压缩

文本压缩是LLM Lingua提供的一个功能，它通过减少文本数据的大小来优化LLM的操作。视频中提到了一个例子，使用LLM Lingua压缩Autogen研究论文，节省了近20,000个令牌，这有助于在具有限制上下文窗口的模型中保持更多的对话内容。

💡Databricks

Databricks是一个用于数据工程和分析的平台，它在视频中被提到，因为它与Autogen的集成。Databricks提供了一个名为DBRX的通用大型语言模型，并且Autogen提供了集成Databricks的示例，这表明用户可以利用Databricks的模型来增强他们的Autogen代理。

💡集成

集成是视频中讨论的一个关键概念，指的是将不同的系统、服务或模型整合到一起，以提供更全面的解决方案。例如，Autogen与Databricks的集成允许用户利用Databricks的模型和Autogen的功能来创建更强大的智能代理。

Highlights

Autogen发布了0.228版本，带来了许多新变化。

新增GPT助手代理，由OpenAI助手API支持，可以利用多种内置工具如代码解释器、文件搜索和函数调用。

助手代理能够自动存储消息历史并根据模型的上下文长度进行调整，称为threads。

助手代理可以生成文件，如图像和电子表格。

介绍了如何设置GPT助手代理，包括创建代理和配置助手。

展示了如何定义助手代理的工具或内置功能，例如代码解释器和文件搜索。

介绍了恢复群聊的功能，允许通过传递先前对话的消息来继续之前的群聊。

群聊管理器新增了resume函数，可以返回最后的消息和代理。

通过示例演示了如何继续终止的对话，包括加载先前消息和使用resume函数。

展示了如何在没有终止消息的情况下继续群聊。

介绍了使用LLM Lingua压缩文本的工具，可以提高LLM操作的效率和成本效益。

示例演示了如何使用LLM Lingua压缩Autogen研究论文的文本。

展示了如何将文本压缩器与Autogen代理集成，以节省令牌并提高上下文窗口的使用。

提到了对.NET开发者的支持，展示了.NET的更新和示例。

介绍了与Databricks的集成，Databricks是一个通用的LLM，提供了新的开放LLM标准。

展示了如何设置Databricks与Autogen的集成，包括获取API令牌和基本示例。

提到了Autogen框架的开放性，允许用户尝试不同的模型和集成。

最后，提到了一个面向初学者的Autogen课程，帮助用户更好地理解和掌握Autogen。

Transcripts

it's probably been a little bit over a

month since the last update that we got

from autogen but we finally got one and

there's a lot to it okay so this is

version

0.228 and as you can see there are a lot

of changes in this update we're not

going to go over exactly all of these

but we'll go over the highlights all

right so for the first update we have

the GPT assistant agent which is an

agent backed by open AI assistant API

and we can use multiple tools such as

the code interpreter the file search and

function calling and what those are are

built-in tools from open Ai and whenever

you use this assistant then you also get

benefits from what they call threads

which automatically store message

history and adjust based on the model's

context length and we can also have

agents generate files such as images and

spreadsheets and if you want to come

visit this page you know they have

examples of function call a code

interpreter and a group chat with the

GPT assistant agent that you can look at

uh but this is pretty simple to set it

up you're just going to create a GPT

assistant agent and the only other thing

here that looks a little different is

you know we have the instructions you

still have the name the llm config but

also the assistant config okay and they

have that right here it's not really

defined yet but when we scroll down

there's going to be different ways you

can Define this and it's going to kind

of be added to the GPT assistant agent

so that you can Define what tools or

built-in functionality you want this

assistant to have for instance they have

the assistant config here to be a code

interpreter here you set up to have file

search and so another example for

function calling is you define a

function this is get current weather and

then you have an API schema which you

use this function get function schema

give it the actual function you want to

use the name and a description and then

in the assistant config uh for the tools

you just say hey I want this API schema

to be the function for the assistant

agent and for this next one I thought

this was pretty interesting this is

resuming a group chat and it may be kind

of how you're thinking right now is

whenever you end a group chat normally

we set a variable which I'll show you in

just a minute Whenever you set that

variable and you terminate the chat so

you're done well whenever you want to uh

have that chat again you can take that

group chat that you had you can

basically get the last messages and then

resume it with another group chat let's

just let's kind of see how they Define

that here and how it works but like I

said we can resume a previous group chat

by passing the messages from that

conversation to the group chat manager

resume function so there's a different

they've added a function to the group

chat manager so the resume function

Returns the last agent in the messages

as well as the last message itself these

can be used to run the initiate chat the

messages passed into the resume function

can be in a Json string or a list of

dictionary messages so here is what we

here's what we want to see an example of

how to actually continue a terminated

conversation so let's look at this

example all right so we had the basic

setup for autogen and then we create the

group chat objects one thing to note

here is it says they should have the

same name as the original group chat

I'll probably come out with an example

of as I explore this a little bit more

but so they have the typical agents here

right there's nothing nothing crazy here

um they have about five five of them so

here we had the group chat right we had

the agents you know this if you've done

group chats before this is nothing

really new then we had the manager so

you say allen. group chat manager give

it the group chat and the llm config

okay this is really nothing new so far

that we done okay but now we want to

load previous messages from a Json

string or messages or like a list

dictionary and so what I think of

they've done is they've used the two

methods that we uh saw above they just

probably went ahead and did that and

then this is this whole line here all

right so I basically just put that in a

quick formatter so the initial message

was to finalize paper um on Arch GPT 4

on archive and it's potential

applications and then um basically this

right here is is all of the

conversations of all of the agents in

the group chat um the user said agree

and then the planner said great let's

proceed okay so then I think they

terminated the they probably terminated

the chat there and so now when we come

back how they set this up is you say

last agent so it looks like manger.

resume gives back uh it gives back two

different it returns two different um

variables so last agent and last message

okay and you give it you say manager

which was the group chat manager up here

you say resume messages equals previous

St so this is the here it's a Json

string okay and then we can say result

equals last agent. initiate chat so

probably the last agent that was part of

that chat we're going to initiate the

chat from there and as you can see here

says great the planner that was the last

one we just saw right so the planner we

come back here the the name was the

planner great let's proceed with the

plan outlined earlier we come back

that's exactly what this is so this is

this previous state is where we ended or

terminated the chat and now we're just

simply resuming it from here and then

they continue the engineer uh creates

some code we come down some more you

know and then now all the agents are

just continuing this chat right so we go

on uh we go on and on and you know we

have U we have the output so it looks

like that was if you just maybe close

the chat yourself because they have also

have an example of whenever you uh

resume a terminated group chat okay so

this is basically whenever uh at the end

you know and one of the agents says okay

terminate the user says okay and then it

cuts out the conversation like we're

done and what this and what this is

doing right now um they're saying that

when they go ahead and say initiate the

chat again with the last message here

right so they resume from the previous

date um this warning is saying the last

message from that previous group chat

meets a termination criteria meaning

there was a termination like the string

was there then this is what we get right

so thank you um so the last message was

from digital marketer to the chat

manager it says BL you know whatever

they did I'm I'm assuming if we go all

the way to the end here yep right

there's a terminate so that's done and

then this time they're going to remove

that message by using the remove

termination string parameter and then

resume so the same manager. resume

except now they have this okay and then

uh after the the digital marketer says

the same thing and there is a termin

there's all the way to the end oh see it

removed it right so this time instead of

you know this is the same text as up

here but they got rid of the terminate

so now we don't know now it's going to

continue the conversation because it's

not looking cuz it's still looking for a

terminate message and then the chief

marketing officer finally comes to

terminate and they're the ones that

actually terminate the conversation now

that's interesting I think that's uh I

think that's pretty interesting so I

think that's a pretty interesting update

let me know what you think are maybe the

use cases of being able to resume a

group chat and potentially extending the

group chat after it terminated which is

what we just talked about okay so for

the next one we have compressing text

with llm lingua and what this is is a

tool designed to compress prompts

effectively enhancing the efficiency and

cost effectiveness of llm operations

okay in this first example we have

compressing their re the autogen

research paper using uh using this

Library they're using this text

compression and so what we have here

just go through this real quick so we

have the the archive link for the paper

this right here this extract text from

PDF just basically Returns the text

within the PDF so we come down here you

know we had the PDF text we instantiate

LM lingua we also instantiate the text

compressor for a text message compressor

object to be llm llm linga and now we

say we want to

apply the text compressor from llm

lingua to the PDF text that we extracted

and then we print um we print the logs

and what this is saying is that we've

saved almost 20,000 tokens right so what

this is going to do is going to save

like the window I know that models you

know what Gemini 1.5 just had what 1

million is or is that the one with two

million context window right so it's not

like the end of last year when I first

started this I remember that uh gbt 3.5

you know we wor about 4,000 tokens right

and we were always constantly trying to

figure out how um how we can fit more

tokens inside of that and what we can

remember how do we preserve

conversations through long-term memory

so I understand that that's not as much

of an issue as it was then but there are

still good models out there that maybe

don't have a great deal of context

window just yet and so you still want to

save you still want to save um tokens if

you can so that you can have more in

your context window because maybe you

need to give it um maybe you need to

give more context that is like you know

say 50,000 tokens right but maybe the

model only holds 32,000 that you want to

use for your use case so something like

this can really still really help you

out okay so that was a quick example now

how do we actually integrate this with

an autogen agent so all of this you know

all of this is the same setup and now

and now we want to add the contact

handling to the researcher agent right

up here right and so this transform

messages with the text compressor that's

um that's what allows us to use LL llm

lingua with uh with this agent so we

just basically uh want to research this

paper include the important information

and then you know we add the context

with the whole PDF right and we don't

and we're going to let the text

compressor handle all of this so the

result you know we just want to initiate

the chat uh the user initiate the chat

with the researcher right here and then

we print um we print the chat history

well it's saying that almost 20,000

tokens were saved and you know it still

describes the paper and gives the key

components right so it's going to help

with the saving the cost on the tokens

per you know whatever million that um if

you're using open AI right how much they

cost so you're still going to save with

that and the context window for whatever

model you're using and then this last

example they just give you ways on how

you can modify see they instantiate the

text message compressor they

specifically say it's llm lingua which

it will I think it is by default

probably um and you can just modify some

of the parameters for this and just one

thing one thing I want to note now

before I get into my next one cuz I'm

not going to go over all these is they

really I mean it seems like they really

are supporting if you're a net developer

I mean look at how many dot uh look how

many net updates there were right these

are all like the commits but I mean

there's there's a good many even though

some of them might just be read read me

readme sections right it's not all these

aren't probably all code changes I

haven't like gone through them so I mean

I could be wrong about that uh like

here's with net here add o llama sample

so those of you who are wondering um if

you can get Lama working locally

with.net here you go they are working on

it and making it happen and just as a

side note in their Gallery also they

have some other you know they update

this so F if you're curious about how

like maybe coding something or how

something works that other people have

worked on and it may give you uh some

insight or inspiration to develop

something similar just come here they've

updated this right so here's a crypto

transactions agent here's a virtual

focus group a um you know this is an

application developed it it's like a

group of Agents with streamlet I believe

so check that out and look somebody here

has created an autogen robot that's

pretty cool right so just come here and

make sure you check this out and the

last one I'm going to talk about is an

integration with data bricks and I think

that you know if if those of you have

used Lang chain one of the nice things

about Lang chain um and they actually

have just updated their documentation

which is better but their documentation

was lacking because they have all of

these Integrations right but you know

things right now the a world are

constantly being updated so some of them

were maybe deprecated and not quite

working like their documentation was

saying so it can kind of make it

frustrating to use but they do have a

lot of ways to make the integration

simpler and one thing that I'm working

on and I actually have a video coming

out is um having using autogen and

integrating other services such as my

next one will be air table which is an

online database among other things um

and then also like a Wikipedia search

but you can do a lot more with it than

what Lang chain does but in this update

we have a data bricks integration so in

2024 they released uh dbrx just I guess

how you would maybe spell data bricks um

a general purpose llm that sets new

standards for open llms they they have

open source models on hugging face so

you can check those out here and what

they have here is examples of

integrating data bricks with autogen so

basically if we just let's just scroll

down here a little bit um so we have the

the first of the setup right so for any

integration right you're going to have

to the the setup so so if you go to they

give you examples of um different ways

you can set it up with your ads

workspace Aur workspace or just straight

from datab bricks host um you have to

get I guess an API token and then you

can just set that up here right so

actually probably take CLE minutes a

couple minutes to set that up they give

you the hello world example which is you

know import look this is this is about

as basic as you can get with an autogen

example right so we have an assistant

agent a user proxy agent and then they

just initiate the chat right so this

isn't like you know this this isn't like

something groundbreaking but you know

data breaks has really grown and I think

I like the idea that we're starting the

ALG is starting to integrate more things

with itself and it's really going to

open up more possibilities right because

I think one of the things that people

are kind having trouble with or just

from kind of what I guess understanding

and my comment section is you know if I

want to do this with autogen how do I do

that right it can be something as simple

as you know um reading a PDF and adding

its context into whatever you want to

ask the agent about with that right and

so being able to introduce other uh

models or other companies and use their

models in here is amazing you know and

then they have a coding a a simple

coding agent so you know they come down

here and they just have um examples from

the data breaks assistant agent and you

know just come here and you can try this

out for yourself right so pretty cool

that that they add this in um so that

there are other Integrations that we can

try with autogen okay I hope you're

excited about this update as much as I

was you know it's just always nice that

you know more tools and more things are

being involved with uh with this autogen

framework because the thing is you know

not everybody wants to maybe run open AI

right maybe somebody wants to try uh

different types of models or just run

things locally or maybe you don't want

to run things locally right maybe you

you like open ai's Assistant API that's

what you want to use well now they're

integrating more with autogen so you can

do that the idea is that this framework

is opening up for more people to try

what you like thank you for watching and

I have a beginner course right here for

autogen that you can understand and get

a better grasp of what it is before you

try these updates thank you for watching

I'll see you next video by