Building a Generative UI App With LangChain.js
Summary
TLDRこのビデオでは、3部作のシリーズの第2部として、Langing Chainを使用して生成型UIアプリケーションの構築方法が解説されています。シリーズでは、生成型UIの概念やユースケース、そしてチャットボットのハイレベルアーキテクチャが紹介されています。さらに、PythonパワードのバックエンドとNext.jsフロントエンドを使用した実装が詳述され、TypeScript開発者向けにReactサーバーコンポーネントを活用したストリーミングUIコンポーネントの実装方法が解説されています。
Takeaways
- 😀 このビデオは、LangChainを使用して生成的UIアプリケーションを構築する三部作シリーズの第二部です。
- 😀 最初のビデオでは、生成的UIの基本概念、ユースケース、高レベルのアーキテクチャについて説明しました。
- 😀 次のビデオでは、Pythonをバックエンド、Next.jsをフロントエンドとして使用するチャットボットを取り上げます。
- 😀 TypeScriptの開発者でない場合は、次のビデオを待つか、今日のビデオの一部のトピックが重複するため視聴することをお勧めします。
- 😀 今日のビデオでは、ユーザー入力、画像、チャット履歴をLMに送信し、条件付きエッジを使用して応答をクライアントにストリーミングするチャットボットの内部構造について説明します。
- 😀 Reactサーバーコンポーネントを使用してUIコンポーネントをストリーミングし、サーバーからUIに戻すためのユーティリティサーバー.tsxファイルを実装します。
- 😀 LangChainのストリームイベントエンドポイントを使用して、すべてのイベントをクライアントにストリーミングします。
- 😀 チャットボットの構築に使用するLグラフエージェントを構築するためのLグラフのビデオへのリンクも提供されます。
- 😀 GitHubリポジトリの詳細を取得するためのツール、請求書の詳細を抽出するツール、天気情報を取得するツールを実装します。
- 😀 最後に、デモ用のアプリケーションを起動し、チャットボットが正常に動作することを確認します。
Q & A
このビデオは何について説明していますか?
-このビデオは、Lang Chainを使用して生成的なUIアプリケーションを構築する方法について説明しています。
前のビデオでどのような概念がカバーされましたか?
-前のビデオでは、生成的なUIとは何か、ユースケース、高レベルのアーキテクチャについて説明されました。
次のビデオで取り上げる予定の内容は何ですか?
-次のビデオでは、PythonチャットボットのバックエンドとNext.jsフロントエンドについて取り上げる予定です。
このビデオではどのような技術を使用していますか?
-このビデオでは、Reactサーバーコンポーネント、Lang Chain、AI SDKなどの技術を使用しています。
サーバーコンポーネントで行われる処理は何ですか?
-サーバーコンポーネントでは、UIコンポーネントのストリーミングと実行可能なランダム処理が行われます。
ストリームイベントとは何ですか?
-ストリームイベントは、Lang Chainアプリケーションからクライアントに中間ステップをストリーミングする方法です。
実行可能なUIを作成するための主要なファイルは何ですか?
-utils/server.TSXファイルが、UIコンポーネントのストリーミングと実行可能なランダム処理のロジックを含んでいます。
GitHubリポジトリツールの役割は何ですか?
-GitHubリポジトリツールは、GitHub APIを使用してリポジトリの詳細を取得し、UIに表示する役割を果たします。
チャット履歴の管理方法について説明してください。
-チャット履歴は、ユーザーの入力や画像とともにLMに送信され、適切なツールを使用して応答が生成されます。
エージェントエグゼキューター関数の目的は何ですか?
-エージェントエグゼキューター関数は、ステートグラフを作成し、ノード間のエッジを定義して、ツールの呼び出しや応答の生成を管理します。
Outlines
😀 シリーズ第2回目のビデオ紹介
この段落では、Braceが3部作のシリーズの第2回目のビデオを紹介しています。シリーズはジェネラティブUIアプリケーションの作成に焦点を当てており、Chainを使用しています。前回のビデオではジェネラティブUIの概念や使用事例、Chatbotのアーキテクチャが説明されています。また、明日公開される次のビデオではPythonパワードのバックエンドとNext.jsのフロントエンドについて触れる予定です。TypeScript開発者でない場合は、次のビデオを待つか、現在のビデオでもNext.jsのフロントエンドやコードのストリーミングに関連するトピックが重なっているため視聴を検討するべきです。
🛠️ 高レベルアーキテクチャのリフレッシュ
この段落では、ビデオの高レベルアーキテクチャのリフレッシュが行われています。Chatbotがどのように動作するかを説明し、ユーザーの入力やチャット履歴をLanguage Model(LM)に送信し、応答を生成するプロセスが解説されています。応答がテキストのみの場合は、クライアントにストリームバックされることが示されています。また、ツールが使用された場合は、ツールの実行とUIコンポーネントの更新が行われます。このプロセスはLangChainと呼ばれるツールを使用して構築されており、条件付きエッジやストリームイベントなどの概念が紹介されています。
📝 utils.server.tsx ファイルの実装
この段落では、utils.server.tsxファイルの実装について説明されています。このファイルは、UIコンポーネントのストリーミングとランナブルの呼び出しに関するロジックを含むReact Serverコンポーネントです。重要なインポートと型の定義、およびwithResolvers関数の作成が行われています。この関数は、プロミスとresolve/reject関数を返すために使用され、UIコンポーネントのストリーミングを完了する際に使用されます。
🌐 エンドポイントの公開とUIコンポーネントのストリーミング
この段落では、APIエンドポイントの公開方法とUIコンポーネントのストリーミングについて説明されています。exposeEndpoints関数を使用してアクションに適切なコンテキストを提供し、AIプロバイダーReactコンテキストを活用してクライアント側のファイルにコンテキストを提供します。また、createRunnableUI関数とstreamEvents関数が実装され、これらの関数はランナブルとUIコンポーネントのストリーミングを処理します。
🔗 ストリームイベントとランタイムUIの実装
この段落では、streamEvents関数とcreateRunnableUI関数の詳細な実装について説明されています。これらの関数は、ランナブルからUIコンポーネントをストリーミングバックし、ツールの呼び出しを処理します。streamEventsは、ランナブルからのイベントをストリームし、UIの更新やテキストのストリーミングを担当します。createRunnableUIは、UIコンポーネントを作成し、ランナブルをラップしてストリームイベントを生成します。
🛠️ ツールの実装とランタイムUIの使用
この段落では、ツールの実装方法とランタイムUIの使用について説明されています。GitHubのリポジトリ情報取得ツールの例を通じて、Zodを使用したスキーマの定義、ocitを使用したGitHub APIの呼び出し、createRunnableUIを使用したUIコンポーネントのストリーミングが実装されています。ツールはダイナミックに定義され、言語モデルから取得されたパラメータに基づいてAPIを呼び出し、UIコンポーネントを更新します。
📈 天気予報ツールとインボイスツールの紹介
この段落では、天気予報ツールとインボイスツールの概要が紹介されています。これらのツールは、言語モデルから得られたパラメータをもとにAPIを呼び出し、UIコンポーネントを更新します。天気予報ツールは、city、state、およびオプションのcountryを受け取り、weather.gov APIを利用して現在の天気情報を取得します。インボイスツールは、アップロードされた画像からインボイスの詳細を抽出し、UIコンポーネントを生成します。
🔄チャットUIの実装とデモ
最後の段落では、チャットUIの実装とデモが行われています。チャットUIはクライアント側のコンポーネントで、サーバー側のランタイムUIを呼び出します。チャット履歴、メッセージの送信、ファイルのアップロードなどが行えるインターフェースが提供されています。また、デモでは、天気予報、インボイスの詳細、GitHubリポジトリ情報の取得など、実装されたツールを使用して様々な機能がテストされています。
Mindmap
Keywords
💡Generative UI
💡LangChain
💡Next.js
💡React Server Components
💡AI SDK
💡Stream Events
💡Runnable Lambda
💡Conditional Edge
💡Multimodal Inputs
💡Agent Executor
Highlights
シリーズの第2弾として、generative UIアプリケーションを構築するための3部作のビデオを紹介
前回のビデオではジェネラティブUIの基本概念やユースケース、チャットボットのアーキテクチャが説明されている
明日公開の次のビデオではPythonパワードのバックエンドとNext.jsフロントエンドを構築する
TypeScript開発者ではない場合は、Next.jsのビデオを見逃さないで推奨
ビデオではReactサーバーコンポーネントとLang Chainを使用してUIコンポーネントをストリーミングする方法が解説されている
Lang ChainとLang Graphを使用して、サーバーからUIへのイベントストリーミングを実装
stream eventsエンドポイントを活用して、Lang Chainアプリケーションからの中間ステップをクライアントにストリーミング
utils server.tsxファイルの解説で、Reactサーバーコンポーネントのロジックを含む
AI SDKのインポートとその使用方法、UIコンポーネントのストリーミングに活用方法が紹介
withResolvers関数の解説で、プロミスとresolve/reject関数の使い方と重要性が強調
exposeEndpoints関数を使用して、アクションに適切なコンテキストを提供する方法
createRunnableUI関数の解説で、ランナブルLambdaとUIコンポーネントのストリーミング方法が説明されている
streamRunnableUI関数の解説で、エージェントの呼び出しとUIコンポーネントのストリーミングを実装する方法
L-graphエージェントの作成と、チャットボットの内部アーキテクチャの解説
invoke modelノードとinvoke toolsノードの役割と実装方法が解説
GitHubツールの実装例で、Zodを使用したスキーマ定義とocit SDKの活用方法
ツールのロジックを実装し、APIのレスポンスをUIコンポーネントにストリーミングする
チャットJSXコンポーネントの解説で、クライアントサイドの実装とサーバーサイドの連携方法
デモアプリケーションの実演と、実際に構築されたチャットボットの機能紹介
Pythonビデオの告知と、今後の公開予定についての案内
Transcripts
what's up everyone it's brace and this
is the second video in a three-part
series on building generative UI
applications with laying chain if you
haven't seen the first video already you
should go back and watch that in that
video we cover some high Lev Concepts
such as what is generative UI we cover
use cases and then also we go over the
highle architecture of how we're going
to build this chatbot and then also in
the next video which releases tomorrow
the python chatbot uh which will be a
python powered back end and nextjs front
end that being said if you are not a
typescript developer you should probably
hold out for tomorrow's video or you can
still watch today's video because some
of the topics will overlapped with the
nextjs front end um and the aisk which
we will use to power some of the uh code
which sends the UI component from the
server or in this video's case the react
server component back to the UI so as a
quick quick refresher this is the highle
architecture of what the chap bot we're
going to be building today will look
like on the inside so it takes some
inputs user input any images some chat
history sends that to an LM the LM has
some tools bound to it then using that
response uh we have this conditional
Edge if you're familiar with L graph you
should know the what a conditional Edge
is if not I'm going to link a uh video
going over L graph which you should
watch because that is what we're going
to be using to build our L graph agent
in this video so the conditional Edge
says if there's only a plain text
response then stream those chunks back
to the client and then as this those
chunks come in from open AI or whatever
model provider we use we render them on
the client if a tool is used that gets
sent to our invoke tool section where we
first stream the initial component back
to the UI which could be some sort of
loading component or UI element that
tells the user hey we have process your
request we're using this tool um and
we're going to get back to you in a
second so it just allows for a quicker
time to First interaction we then can
execute some arbitrary tool function
which is just a generic typescript
JavaScript function so in our case it'll
typically be hitting an a an external
API and then once we get the response
back from that we update our UI
component stream with the final
component and close it uh you can call
update or append as many times you would
like to update or append UI components
um to the UI your user see
sees the way we're able to stream all
these events back to the client is via
the stream events endpoint in Lang chain
stream events essentially is a way to
stream intermediate steps from your
Langan chain application back to the
client or just send them in a in a
response object um since this is all
using Lane chain and Lane graph stream
events is able to access every single um
yield that a function might yield so in
our case we're going to be yielding UI
components we're going to be yielding
data and then once again yielding UI
components again or yielding text um in
stream events is able to just capture
all of those streams and then forward
them back to the client okay so the
first file we're going to want to
implement is our utils server. TSX oh
and if you want to follow along the link
to this GitHub um repo will be linked in
the description you can clone it and
then you can go through and read each of
the files as we as we code them um the
code will all be there in that refo but
this first server. TSX file this file is
what's going to contain all of our logic
around streaming UI components and
invoking our runnable so let's add our
Imports the first line you can see it's
server only the all of the streaming UI
components and invoking the runnable
will have will happen inside of a react
server component so we want to make sure
this file is only used on the server
next we're going to import react node
and is valid element from react we're
going to use uh the this for typing as
this is a type and then is valid element
we're going to use to make sure the
elements inside of our stream from our
runnable are UI elements before sending
them back via the aisk which leads us
into this next import the AIS SDK
Imports um we use the AI SDK under the
hood to handle streaming back UI
components because they do a lot of the
heavy lifting with react server
components next we have some imports
from runnables or sorry from Lang chain
core runnables uh this will we're going
to be using as a type and then also
runnable Lambda is what we're going to
use to wrap our one of our streaming UI
components back so we can um essentially
upsert the UI components into the stream
event next these are all going to be for
Ty types we have our stream event type
and then we have our AI provider which
is going to provide context we're going
to use this to wrap um our children so
that all of the UI components have the
proper context and then this we're also
going to use as a
type so the first function we're going
to implement is the with resolvers
function this function is going to
return um a promise and then a resolve
and a reject function we're going to use
these so that when you are streaming
back from the UI component or sorry from
when you're streaming the UI component
and any other values from our chain
we're able to properly resolve the final
promise and know when our UI is done um
streaming inside of this component we
have a resolve reject function a new
promise which then assigns the resolve
and the reject function to the resolve
and reject function from the promise and
then we return it we need to expect the
error here because typescript does not
think that these have been assigned
technically that true however the way
we're going to use this um we will
always use this promise
before calling these resolve and reject
functions so although typescript thinks
they're not used yet in practice we will
use them after calling our
promise next we're going to implement
our expose endpoints function this
function is what we're going to use to
return the proper actions and provide
context to these actions so they can
call our agent and then also so our UI
components have the proper context we're
using this AI
provider react context that we imported
from client. TSX which is is in which is
in the same UTS file if you're following
along and then as we see here we also
have this use context hook which uses
the use context from react and it's
going to provide context to our client
side uh files so they can properly
invoke the
agent here it's pretty simple returns a
new function AI which contains an AI
provider um react contacts or jsx
component which you saw in the other
file the next function we're going to
want to implement is the one which will
handle streaming back all the UI
components so handling this stream
handling this stream and then it's what
passes the UI components from our tool
call functions up into our stream events
call so we'll implement the stream
events function after this one but this
function we're going to implement will
handle we will
is essentially wrapping the aisk and a
runnable Lambda and is uh streaming and
yielding these UI components so that we
can access them inside of our stream
events we'll Implement that here it's
going to be called create runnable UI
takes in two args one is required the
second is optional the config argument
this is going to be used to we're going
to pass in our config config values here
and this is so these stream events um
when we invoke stream events it's going
to have access to this runable Lambda
function next we take in a an optional
initial value this gets passed to the
create streamable UI function so you see
right down here we have our runnable
Lambda this Lambda takes in a single
input of initial value which should
actually be
optional um this initial value is going
to get passed to the create streamable
UI function from the asdk and then we're
going to return this UI value this UI uh
function is going to have a value which
is the jsx element this is what we're
going to use to actually send back to
our client and it's going to contain the
stream which can update uh anytime we
call update aen error done um and this
is what we're going to render on the
client we then attach a config so that
this runnable Lambda always has the same
name stream UI Lambda and this is what
we'll use later on to identify this
runnable Lambda and extract the UI value
from it and then we're going to return
us invoking that function passing in the
proper config so that using the we pass
in the config so that stream events is
able to find this runnable Lambda inside
the St stream
event the next function we're going to
want to implement is the function which
we'll we'll use to invoke our agent this
is also going to call stream events and
extract any UI values we return from
here so when we invoke our agent it's
going to be using calling stream events
inside of this function so let's paste
that in and then let's walk through
exactly what this does
so it's called stream runnable
UI the first line we're creating a new
streamable UI from thek and then using
that with resolvers function we
implemented below uh the last event
which is our promise and then our
resolve function we then have an async
function which executes in here uh we
have some callbacks which contain a
string and then either the return type
of create streamable UI or create
streamable value we'll use those in a
second and then as you can see our first
input is a runnable
we're going to call stream events on
this runnable to stream back all the
events this runnable will be our
agent we iterate over each event and the
first thing we do is we check to see if
it's a UI value which was returned from
this Lambda so as we can see our run
name is here we're checking to see if
the stream event. name is that run name
and it's after that Lambda has finished
so on chain end should be the event if
it is that event and and runnable Lambda
we're then going to check and make sure
that the value of the output is a valid
UI element if it is then we're going to
append that UI element to our UI which
we created via the great streamable
UI next we're going to check to make
sure that it's a stream and it's not a
chain um this is what we're going to use
to extract any text values from the uh
from the our agent or our Ling graph
graph that will be this part of our
diagram where the LM returns just text
I'm going to send it right
back so if it
is text and type of chunk. text is
string we want to make sure we have not
already processed this this run run
event because your llm could in theory
invoke a language model twice and get
two sets of text streams back so we make
sure that we've not already processed it
if we haven't then as this comment says
the create streamable value sluse
streamable value is preferred as the
stream events are updated immedi in the
UI rather than being batched by react
via create streamable UI so if we're
just updating text we want to use this
create streamable value and not create
streamable UI then we're going to append
our create streamable UI with a generic
react or jsx function this could be you
know this is customizable for what you
want we have ai message text which is
going to be our text bubbles but you
should probably replace this with
whatever UI you want and here you say
you see using Create streamable value
which we mentioned there and then the
value of that is the value of our text
stream and we're using text stream so we
can bypass any sort of batching that
react does and instantly update our UI
with the stream as it comes
in next if the Run idea is true which it
is because we just said it we're going
to append the text from the stream event
chunk this will be the text language
while stream back and then we're also
going to be updating our last event
value of the stream event this happens
at the end of each stream event so we
know that the last event value will
always be the last stream event finally
when our stream events has resolved
we're going to resolve our promise which
we implemented in the with resolvers
function with the output of our stream
event this will typically be some sort
of string but it could also be um an
object with say our tool
call we're then going to iterate over
all of our callbacks and call done on
them to finish our callbacks and then
call UI Doone which is going to close
the UI stream between the server and the
client via the the aisk finally we're
going to return these values in the last
event which we will use in our client
when we Implement
that now that we've implemented our
stream runnable UI and create runnable
UI we can go ahead and Implement our
graph this is going to be this language
model graph with the Edge invoking tools
or sending back the response we're going
to go into the AI graph file and first
things first we're going to add our
Imports as you can see we're not adding
any serveron um text because since this
is only going to be used inside of our
serveron code here it'll already be
server only we're importing some prompt
templates um start and end uh variables
from L graph State graph which is what
we're going to use to create our lane
graph chat open AI you can obviously
replace this with any language model
which supports tool calling from the the
lane chain Library um our GitHub tools
which we'll Implement later or after
after this I guess we're going to find
our base message which we use for types
and then runable con config which we
will also use for types once we've added
our Imports we're going to want to add
our type for our L graph agent so we're
going to name it agent executor State
the first value is a is the input this
is going to be the input that the User
submitted right here next is chat
history once again the chat history from
their previous conversations and then we
have some optional values these are
optional because they're only going to
be popular ated later on in our graph so
result the plain text result in LM if no
tools used that's going to be this part
so if the LM does not invoke a tool and
only returns some text that's going to
populate this value next is the parse
tool result that was called if the LM
does call a tool we're going to parse
that and return it that will be pointing
to this conditional Edge and then
finally the result of a tool that will
be anything that this arbitrary function
returns um and we're going to uh
actually include the result of that
because we want to update that in our
chat history later on so the language
model knows that it did in fact um
invoke and complete any tool requests
that the user ask
for after that we can skip adding our
nodes for now because we're want to con
we're going to want to construct the uh
graph
first so we're going to create our agent
executor function this is going to
create a new state graph passing in our
state and then also creating these
channels um here we're going to add just
two nodes one for invoking the model and
one for for invoking the tools invoke
model will obviously be this part make
that a little bigger and then invoke
tools will be what happens when we pick
a tool and we want to invoke that tool
invoke model will then always call
invoke tools or return which is this
conditional Edge this conditional Edge
will just check to see if the tool is
used if it is used then this function
will then return invoke tools so invoke
tools is called if it's not used then
it's going to return end which is this
end variable and that indic Ates to Lane
graph that it should always finish and
that's what this is where it finishes
and sends the response back invoke tools
will also then always end so that once
the tool is done invoking it returns and
finishes the L graph graph and responds
back to the UI and obviously start is
always going to call invoke model
because that's the first thing we want
to do in all of our graphs we're then
going to compile it and return our graph
and then we're going to use this a
little bit later on when we're passing
it to our stream runnable UI
and this graph right here will be the
runnable that is in is invoked via
stream
events so if we go back to our graph
file the first node we're going to want
to implement is the invoke model as
that's going to be the first node which
is always called so if we paste that in
let make this a little bit bigger we can
see it takes in two values state which
is our agent executor State this is kind
of the magic behind Lang graph where
it'll always pass the state to every
single node even though we're not
returning the full State here so Lane
graph can recognize what values in your
state that you returned it just appends
that to the total State and then passes
the complete State through to each node
so we take in our state and then also
our config value this is what we're
going to pass to invoke so that out our
Lang Smith traces and the stream events
all have the same all contain the same
runs in the single trace so stream
events can get back all the
values first thing inside our functions
we Define our prompt you're a helpful
assistant you given a list of tools need
to determine which tool is best to
handle the user input or respond with
plain text we then have a message
placeholder for the chat history this is
where our chat history rle go so the
language model has access to all of our
history it's obviously going to be
optional because there will be no
history on the very first invocation and
then we have our human message with just
a plain input uh
argument next we Define our tools we'll
Define those after this function but
we're going to provide three Tools in
language model uh GitHub tool invoice to
web weather tool when we Define these
tools we'll talk about what each of them
do next we're going to Define our llm
we're not going to give it we're going
to give it a temperature of zero so it's
more predictable and not
as um creative I guess we're going to
use gbt 40 because that's their newest
fastest model which can give us text
back super quickly and also process our
images um and then we're going to bind
tools to this model so all of the the
model has access to all the tools we've
defined next we're going to use the line
chain expression language anguage to
pipe our prompt to our language model
and create a chain and then we're going
to invoke our chain passing in our input
from the user input and the chat history
and then also our config object once
this is finished invoking we're going to
check to see if any tool calls were on
the model or if the model uses any tool
calls and if they are we're going to
return this tool call Value which will
populate this
field if the model did not use any tool
calls we're just going to return the
content now that we've defined our first
node we're going to want to Define our
conditional Edge which will always be
called after this
node invoke tools or return this takes
in the state and it essentially says if
tool calls are defined then you want to
call the invoke tools node next and if
it's not defined but the result field is
defined then we're going to end because
that's just the string that was returned
and then this should never happen but if
for some reason neither of these are
defined um then it's going to throw an
error but we're never going to get this
so that should not cause an issue for us
finally we're going to Define our last
node invoke
tools this takes in the same input
arguments as the invoke model our state
and our config um and it's going to
first check to make sure there's a tool
call once again this should never happen
because it should only call invoke tools
if tool calls are defined but because of
typescript we need to add this here but
we should never see this error next
we're going to Define our tools map
which is going to be a map containing
key value pairs each each key is going
to be the name of the tool and then the
value is going to be the actual tool
using this map and our tool input we're
going to try and find the tool in there
once again this should never happen
because our language model especially if
you're using a um state-of-the-art
language model it should never pick a
tool which doesn't exist you know that
you didn't provide to it um but we have
this here once again for typescript once
we have our selected tool we're going to
then invoke that tool passing the
parameters that the tool called for that
the language will provided to us in our
config and then finally once we get a
result back this result will always be a
string but usually we're returning an
objects we're going to parse that result
and then return it in our tool result
value which will populate this
field now that we've done this we've
implemented our entire agent and we can
go and Implement our tools which we will
then provide to the agent these tools
are going to contain all the logic
around streaming back UI components and
hitting any external
apis so for our tools we have this tools
folder which contains some files uh get
a repo weather and invoice we're just
going to implement the get a repo tool
and then I'll quickly walk through the
other tools because they're all pretty
redundant and contain kind of the same
logic uh but for for our GitHub repo
tool we're going to want to add our
Imports first we're going to be using
Zod Zod is what we're going to use to
define the schema so language model
knows what parameters to pass or extract
from the input and then pass to our tool
um ocit which is the GitHub API wrapper
SDK which is is what we're going to use
to actually call the giab API create
runnable UI which we defined in our
server. TSX file which is going to wrap
that Lambda create a new streamable UI
and that's what we're going to use to
actually stream back these UI components
our tool from Lane chain and then our
pre-built components which we can
quickly look at we have a loading
component which just contains some
skeletons to show that we're loading and
then the actual component which contains
a card um this card is going to have a
link to the GitHub repo it's going to
show how many stars they have the repo
description and other things like that
which we'll get back from the GitHub
API the first thing we're going to want
to do is Define our schema our schema is
can to be the owner in the repo which is
the which are the fields that the giab
API requires in order to fetch details
about a repository um if the language
model sees that a user is submitted an
input with an owner and repo fields that
look like a get of repo it'll likely
call this tool and provide us the name
of the repository and the repository
owner next we're going to Define our fun
which we'll actually call the giab API
we're going to call it the giab revo
tool the input is going to be the type
of our Zod schema so z. infer type of
our schema and that will infer the type
here next we're going to make sure you
you have your GitHub API token in your
environment if it's not we're going to
throw an airror obviously if you're
going to use this tool you should set
that in the read me of this repo I add
instructions on how to get all the API
Keys you need for free for the different
tools um except for your language model
API key which will obviously cost money
uh when you invoke the language model
we're then going to instantiate our octo
kit SDK passing in our GitHub token and
that's going to return an instance of
the GitHub client then we can just call
our GitHub client calling the repos um
with a get request and that's going to
get us the information on this repo and
then we return the data from the um from
the API response which is the input so
we have the owner and the repo and then
the get
repo description how many stars they
have and the primary programming
language if there's an error we're just
going to return a string and then we'll
process this inside of our tool to
return either the GitHub final component
or an error component if an error was
occurred now we can Implement our tool
it's going to be a dynamic structured
tool with a name GitHub repo a
description tool to fetch details of
GitHub repository um we're going to pass
in our schema so language model knows
what fields to provide and then we have
our function inside this function we see
it takes two arguments input which
should be the schema we defined and then
config we're going to first create a new
runnable UI stream passing in
our initial value which is going to be
this loading component and this is going
to tell the user that as soon as the
language model picks this tool and this
function is invoked it's going to the
user is instantly going to get back
their
first loading component so they see that
we're working on something that's this
step here next we're going to hit our GI
of API with this function we defined
above passing in our input then if they
get of API return to string there's an
error and we're just going to return um
a P tag with the error message calling
stream DOD if the type was not a string
so this object here then we're going to
return our GitHub component which is
that jsx component which will actually
show all the data passing our data and
finally we're going to return the result
of all result of our tool you can also
call stream. update or append as as many
times you would like if you want to say
hit an API update your tool hit another
API add some more values to that UI
element you can really call update um
and append as many times you would like
to keep interacting or updating that
interactable UI component with the user
the nice thing about this as well is
since it takes in a react node our
GitHub component these don't have state
however they could be stateful they
could contain some button which hits an
API or makes another language model call
which then updates the UI again and
they're really just generic react
components so everything that you could
do before with react components and make
them super Dynamic and interactable you
can do that here as well because it's
any sort of react component you want and
you can pass props to it so they can
take in these different inputs and be
dynamic and customizable for the user so
now that we've imped these tools we can
quickly look at our invoice and weather
tool invoice is just a schema this is
because we're going to want the language
model to extract these fields from any
uploaded image and then it just creates
a own UI with this initial loading which
is kind of redundant because then it
instantly turns around and um updates it
with the final component but just to
show the same thing and then it Returns
the input and then for the weather
component or for the weather tool same
thing our weather schema is a city and
state and then an optional country which
which it defaults to USA and then it
hits a couple apis if you want this API
key I've added some instructions in the
REM on how to get it it's totally free
um it's going to get the uh longitude
and latitude from this API and then it's
going to use the weather.gov API which
is free passing in these values and then
it's going to extract the current
weather for your location the tool also
the same thing creates a stream passes
back that loading weather component then
it invokes our weather data function to
actually get the weather data from these
apis and finally it updates the weather
component with our um with the data that
the API returns
finally we're going to want to implement
our chat jsx component which is going to
be the chat you can interact with if you
want to follow along you should go to
pre component prebuilt chat this going
to be a client component because it
doesn't is not actually uh using
anything in the server instead it's
going to call our server component we're
going to have a state input which are
just some sh Shad CN components that
they built uh endpoints endpoints
context which we already defined
here or sorry we need to Define this
after this um our use actions which we
saw how that was defined that's going to
provide the agent act action for us and
then our context which is going to
provide context to our action and also
um render our UI
elements these are just some util
functions on converting files to base 64
we need to convert them to a Bas 64
string on the client because react
server components don't allow for any
arbitrary function fun to be passed over
the or sorry any arbitrary object to be
passed over the wire only specific
objects and obviously uh key value pairs
where they're both strings and we would
need to convert it to Bas 64 in the
server anyway so we just do it on the
client U because they don't allow for
file objects to get passed over the wire
so it's now string and we send it over
and then we use it to invoke our
language
model for our chat function we're going
to want to add our state variables first
our use actions hook providing our
endpoint cont text which is our agent
takes in our inputs which we also see
here this is how we're going to then
invoke our agent we then have a few
State variables Elements which is a list
of jsx elements these are going to be
the elements that the UI returns to be
rendered uh your chat history chat
history input and then a selected file
you've selected here we see we're
wrapping our elements with this local
context. provider um context jsx uh
element and that's going to provide
context to our react elements
and then we have a simple form as we
would see right here just for submitting
your inputs and any uh images you might
upload next we're going to want to
implement our on submit function which
is going to actually call our
agent so this on submit function does a
few things we're going to paste it in
and then walk through it so first it
makes a copy of our elements array so in
case this somehow gets updated later on
it's not going to mix and match them so
it's always going to be the same at the
beginning of the function it's then
going to convert your file to base 64
string format if you did upload one and
then it's going to use our action our
agent action to invoke our agent this is
going to return an element which is UI
and last event we saw that here it's
essentially calling this function which
returns our UI and last
event passes any necessary inputs and
then updates our element array with the
UI value from our return value from our
agent and then also so the human message
that the User submitted and then if they
uploaded a file the
file finally we saw in our server
function the first function we
implemented was was this with resolvers
we're going to use that here so element.
last event which once again is
the last event here from with from our
with resolvers
function it's then going to check and
see if it's an object this right here is
specific to this chatbot we've
implemented so you're obviously going to
want to update these to reflect your L
graph node but we have or nodes we have
invoke model and invoke tools as we saw
we defined in our graph so if it's an
object then we're going to want to see
if last event. invoke model. result is
true that would be this plain text
string if it is then we update the
history with that plain text and if it's
not true that means that tool was used
so we see invoke tool
and then we update our history um but
make the assistant message be this tool
result and then the result of our tools
and that's so the the assistant knows
it's it's successfully completed the
request we made in the past using the
result of our tool API um and that's so
you can say something like uh you know
what's the details on this GI up repo it
sends the response it didn't actually
give you any text there wouldn't be any
text in the chat history but since we
are creating this assistant message when
you send a followup it's able to see
your question and see that it was
submitted or resolved and in those it
should ignore any questions in your chat
history because it's already resolved
them here finally we clean up by setting
our elements State value and then
clearing any
inputs finally we can go and Implement
our agent wrapper which is going to uh
use this stream runnable UI function and
call our graph agent executor function
um and and that's what we're going to
use to invoke our agents let's go
Implement that now so if you're
following along you should go to app SL
agent and you're going to want to paste
in our Imports once again server only
because we're going from this client
component to our server so we want to
make sure it's only invoked on the
server we're then going to import our
agent executor from our graph which we
built a little while ago and our expose
endpoints and stream runal UI from our
server file which we invoked in the
first part finally we're going to want
to import our AI message and human
message uh Lane chain message types
we're going to use this when we're
constructing our chat history to give
the proper message types to our um chat
history and this will also when we look
at the Lang graph Trace you'll be able
to see human message AI message human
message AI message um in the proper
types uh so the language W knows what or
who said
what now we can implement this little
util function for converting our chat
history type see from chat roll and
content into a proper list of human
message and AI message so iterate over
if it was a human then return a human
message or is thiser AI return an AI
message and then by default we're
returning a human message this could
also be say chat message from linkchain
core and this would just be a generic
um we also need a roll so you could say
you
know roll um and this would just be a
generic chat message and it wouldn't be
specifically human or AI but in our case
this will probably never happen because
we know we're always adding this role
and assistant but we're adding it just
to make our switch case
happy next since we're doing image um
inputs we need this process file
function this is essentially going to
process any files you upload and convert
them to the proper message type for
passing multimodal inputs to our
language model so if file is defined
it's going to create a new human message
um passing into content which is a list
of Type image URL and then image URL
with our base
64 uh conversion of our file and this is
the right or this is the proper uh
template format for uploading multimodal
types to language models I'll add a link
in the description to our multimodel
how-to guide in the JS documentation if
you're interested in that and then
finally we return our input and our chat
history which matches as we saw the
input in chat history from our Lan graph
agent if you didn't upload a file then
we just need input in chat history and
don't need to do anything with image
prompt
templates now we need to implement our
agent function this is going to be the
function that when you
call where is it actions. agent um this
is going to be the function that's going
to that it's going to invoke so it's
going to take in as we see the same
inputs we were passing in here input
chat history and file
um use server so we make sure that this
is always executed on the server as in a
react server component we're then going
to this does not need to be a
sync um we're then going to process the
file to get the process inputs which is
input in chat history that's going to do
what we saw up here and then finally
it's going to return a new stream
runnable UI passing in our Asian
executor which is our Lang graph agent
and any inputs um we saw this already
but this is just going to call stream
events on our runable which we pass into
it which is our agent executor function
and the last thing we need to do is
actually create uh expose the context
for this agent so that our actions.
agent um and with our use actions hook
has the proper context available to
invoke
this so as we see here there's an error
because exposed context does not exist
yet but once we expose our context
passing in our agent function and Export
this the goes away and as we
see with our actions. aent it has access
to our agent function with the same
inputs we just defined here we see we're
passing expose uh endpoints this agent
function with the same inputs and this
just gives the proper context to um our
CL C client component so it knows that
it can invoke this agent now that we've
implemented all this we can go and
actually demo our application
restart
go now that we've implemented all this
we can go and start our Dev server and
actually check out our demo so we're
going to want to navigate to your
terminal go into the proper directory
and run yarn Dev or you can build it run
yarn start kind of the same once it's
running we're going to go to Local Host
3000 and we will see our website so
gener VII with L chain we see our chat
bot and our inputs um we can say
something like what's the weather in
SF it then streamed back our text if we
go back here we can see that the
language model decided did not have
enough um inputs from the user to select
the weather tool or any other any of the
other tools so it just sent back some
text but since we've implemented chat
history can it it said can you please
specify the state so we can just say
California and it's going to use our
chat history and our current message to
then invoke the weather tool and we see
right there there's a loading component
because it picked the weather tool sent
us back our loading component and then
went and actually hit the weather apis
and then updated this component to show
the actual weather we can also say
something like what's the deal with my
invoice and we can upload an image let's
say we upload our receipt we submit that
and this is going to use GPD 40's
multimodal capability to read our image
and then send us back this nice fully
interactable if you implemented this
component it wasn't just a demo um uh
receipt component or invoice component
um and this is all populated with the
receipt image I uploaded so it extracted
the fields and then passed in the proper
properties to this component so it could
render and then finally we can check out
our GitHub component which we
implemented so what the info on
Lang chain AI SL Lang graph and our
language model is going to be able to
recognize that this is a in a get up
repo so we submit that and then error
doing that so I wonder if I spelled
something wrong yes I spelled Lang graph
wrong try Lang chain AI
SL Lang
graph so as we saw there it gave us our
loading component but then the get of
API returned an error so it just
responded with this string when I did it
properly it gave us the proper component
and then obviously it's fully
interactable so you can click on this
and it'll bring you to the L graph
repository that's it for this video um
if you're interested in the python video
it's going to come out tomorrow or if
it's already released then we will link
in the description where we will
implement this exact same uh chatbot
demo website but with a full python
backend I will see you all in the next
video
تصفح المزيد من مقاطع الفيديو ذات الصلة
5.0 / 5 (0 votes)