Introspective Agents: Performing Tasks With Reflection with LlamaIndex

Summary

今日のビデオでは、自己内省型エージェントについて解説します。自己内省型エージェントは、タスクを実行しながら自己反映を行い、最初の応答を反映し、その後、そのタスクのパフォーマンスを評価し、修正するものです。このプロセスは、最終的な応答が満足できるまで繰り返されます。自己内省型エージェントは、タスクを委譲する2つのサブエージェントを含んでおり、1つは初期応答を生成し、もう1つは反映と訂正のサイクルを実行します。また、このパッケージには、ツールインタラクティブ反映と自己反映という2つの反映メカニズムが含まれています。ツールインタラクティブ反映では、外部ツールを使用して反映を行い、自己反映では、事前トレーニングされた言語モデル自体を使用します。ビデオでは、ポジティブAPIを使用してテキストの毒性スコアを取得し、その毒性を減らす方法を紹介しています。結果として、自己内省型エージェントは、タスクのパフォーマンスを大幅に向上させることができ、特に適切なツールを使用した反映がより優れた結果を生む傾向があることが示されています。

Takeaways

• 🧐 内省型エージェントとは、タスクを実行しながら自己反映を行い、初回応答を反映し、その後、その応答の質を評価し、修正するエージェントです。
• 📚 内省型エージェントは、タスクを遂行する際に、タスクに応じた初回応答を生成し、その後、反復的な反映と訂正を通じて最終的な応答に至ります。
• 🔧 「llama index agent introspective」パッケージを使用して、内省型エージェントを実装し、その主なクラスを通じてどのように使用できるかを学びます。
• ⚠️ 毒性削減タスクには、攻撃的または不快なコンテンツが含まれる可能性があるため、閲覧に注意が必要です。
• 📉 毒性削減のタスクでは、ポジティブなテキストを安全で_less toxic_な方法で書き換えることを求められます。
• 🤖 内省型エージェントは、タスクを委譲するデリゲート型のエージェントであり、メインエージェントと反映エージェントの2つの異なるエージェントを含んでいます。
• 🔄 反映エージェントは、反復的な反映と訂正サイクルを実行し、停止条件が満たされるまで処理を続けます。
• 🛠️ ツールインタラクティブ反映エージェントは、外部ツールを使用して反映を行い、毒性スコアを取得する例としてPerspective APIが使用されます。
• 🤔 セルフ反映は、外部ツールを使わずに、事前トレーニングされた言語モデルの知識だけで反映と訂正を行います。
• 📝 クリティックペーパー「Critic」の結果に基づいて、適切なツールを使用した反映は、セルフ反映よりも優れた結果を出す傾向があります。
• 📊 統計的な意味で結果を比較すると、ツールインタラクティブ反映はセルフ反映よりもわずかに低い毒性スコアを生成することが示唆されています。
• 📚 このノートブックでは、内省型エージェントを使用してタスクを実行し、反映と訂正サイクルを通じて改善されたテキストを生成する方法を学びました。

Q & A

イントロスペクティブエージェントとは何ですか？

イントロスペクティブエージェントは、タスクを実行する際に自己反映を行うエージェントです。タスクに対して初期応答を生成し、その後、その応答を反映して修正を繰り替え、最終的な応答に至るまで繰り返しを行います。

自己反映とはどのようなプロセスですか？

自己反映は、エージェントが与えられたタスクの初期応答を振り返り、その性能を評価し、必要に応じて修正を行うプロセスです。これにより、より適切な応答を得ることができます。

LLaMAインデックスエージェントとは何ですか？

LLaMAインデックスエージェントは、自己反映を用いてタスクを実行する新しいパッケージで、特に毒性削減タスクに使用されています。

毒性削減とは何を意味しますか？

毒性削減とは、潜在的に有害なテキストを安全で問題のない方法で書き換えることを意味します。これにより、テキストが持つ毒性を軽減し、より安全なコンテンツを作成することが目的です。

ツールインタラクティブリフレクションエージェントとは何ですか？

ツールインタラクティブリフレクションエージェントは、外部ツールを使用して自己反映を行うエージェントです。たとえば、テキストの毒性スコアを取得するためにPerspective APIなどのツールを使用します。

セルフリフレクションとは何ですか？

セルフリフレクションは、外部ツールを使わずに、事前にトレーニングされた言語モデル（LLM）自身の知識を使用して反映を行う方法です。

毒性スコアとは何ですか？

毒性スコアは、テキストがどの程度有害かを示す指標で、パーセント値で表現されます。0から100までのスコアで、低いスコアは毒性が低いことを意味します。

Perspective APIとは何ですか？

Perspective APIは、テキストの毒性スコアを計算できるAPIです。Google Cloudプロジェクトで有効にし、APIキーを取得して使用する必要があります。

critiqueメカニズムとは何ですか？

critiqueメカニズムは、自己反映を実行する際に使用されるプロセスで、外部ツールのフィードバックを活用してテキストを修正します。critiqueは、critique大型言語モデルがツールインタラクティブなcritiqueで自己訂正ができるという研究に基づいています。

イントロスペクティブエージェントがタスクをどのようにデリゲートするのですか？

イントロスペクティブエージェントはタスクをメインワーカーエージェントとリフレクティブエージェントの2つのサブエージェントにデリゲートします。メインワーカーエージェントは初期応答を生成し、リフレクティブエージェントは反映と修正の繰り返しを行います。

自己反映とツールインタラクティブリフレクションのどちらが優れていますか？

critique論文の結果によれば、適切なツールが使用可能であれば、ツールインタラクティブリフレクションの方が自己反映よりも優れた結果を出す傾向があります。

このノートブックの目的は何ですか？

このノートブックの目的は、イントロスペクティブエージェントを使用してタスクを実行し、特に毒性削減タスクを通じて自己反映を活用する方法を紹介し、そのプロセスと結果を解説することです。

Outlines

00:00

📘 内省的エージェントとは

ビデオでは、テキストの毒性を減らすために内省的エージェントを使用する方法について説明しています。内省的エージェントは、タスクを実行する際に自己反映を行い、初回の応答を反映し、その後、それを修正することで最適な応答に至れます。また、このプロセスに必要なパッケージをインストールし、イベントループを実行する必要があります。さらに、ビデオには、一部の視聴者が不快感を覚える可能性のある内容が含まれているため、警告が発せられています。

05:01

🔍 内省的エージェントのしくみ

内省的エージェントは、タスクを委譲するエージェントで、メインエージェントと反映エージェントの2つのエージェントを含んでいます。メインエージェントは初回の応答を生成し、反映エージェントはその後、修正サイクルを繰り返して最終的な応答を作成します。このパッケージでは、ツールを用いた反映と自己反映という2つの反映メカニズムが実装されています。

10:04

🛠️ ツールインタラクティブな反映エージェントの構築

ツールインタラクティブな反映エージェントは、外部ツールを使用して反映を行います。ここでは、テキストの毒性スコアを取得するためにパースペクティブAPIを使用しています。APIの使用には、Google Cloudプロジェクトで有効化し、APIキーを取得する必要があります。カスタムクラスを作成して、テキストの毒性スコアを取得し、そのスコアを用いてテキストを改善していきます。

15:05

🔧 イントロスペクティブエージェントのテストと自己反映

作成したイントロスペクティブエージェントを使用して、有害なテキストを改善しました。ツールインタラクティブな反映エージェントでは、パースペクティブAPIを用いて毒性スコアを取得し、自己反映を使用する場合は、外部ツールを使わずにLLM自体の知識を用いて反映を行います。結果として、毒性スコアが下がることが確認でき、テキストが安全で Became safer.

20:06

🤖 イントロスペクティブエージェントの応用

自己反映とツールインタラクティブな反映の2つのメカニズムをテストし、毒性スコアを比較しました。クリティックペーパーの結果と一致し、適切なツールを使用した反映は自己反映よりも優れているとされています。また、外部ツールからのフィードバックがタスクの遂行において非常に重要であることが示されました。

25:06

📚 イントロスペクティブエージェントのまとめ

ビデオは、タスクを実行する際に自己反映を行うイントロスペクティブエージェントの紹介を通じて終了しました。このエージェントは、メインワーカーエージェントと反映エージェントにタスクを委譲し、毒性スコアの低減に成功しました。また、ツールを使用した反映が自己反映よりも優れているというクリティックペーパーの結果にも忠実に沿っていました。

Keywords

💡自省的エージェント(introspective agent)

自省的エージェントとは、タスクを実行する際に自己反映を行いながら行動するエージェントです。このエージェントは、与えられたタスクに対する初期応答を生成し、その後、その応答を反映的に評価し、改善を繰り替えることで最終的な応答を導き出します。このプロセスは、テキストの毒性削減タスクにおいて、まず安全でないテキストを入力として受け取り、それを安全でない表現を減らした形で再構築することで、毒性を削減に努めています。

💡反映(reflection)

反映とは、エージェントがタスクの初期応答を生成した後、その応答を再評価し、改善を加えるプロセスです。このプロセスは、テキストの毒性を減らすために用いられ、エージェントは生成されたテキスト草案を反映的に評価し、毒性スコアに基づいて修正を繰り替えます。

💡毒性削減(toxicity reduction)

毒性削減は、ポジティブな対話を促進するために、有害または不快なコンテンツを減らすプロセスです。このビデオでは、自省的エージェントが与えられたテキストを安全でない表現を減らした形で再構築することで、テキストの毒性を削減するタスクを行っています。

💡修正(correction)

修正は、自省的エージェントが行うプロセスで、テキスト草案を反映的に評価した後、毒性スコアを下げるためにテキストを改善するステップです。このプロセスは、毒性スコアに基づいて繰り返し行われ、最終的に安全で適切なテキストを作成することを目指します。

💡Perspective API

Perspective APIは、Google Cloudプロジェクトで有効にできる外部ツールで、テキストの毒性スコアを提供します。このAPIは、自省的エージェントがテキストの毒性を評価し、改善するための重要な役割を果たします。

💡自己反映(self-reflection)

自己反映は、外部ツールを使わずに、エージェント自身の知識と事前に学習された情報に基づいて反映と修正を行うプロセスです。この方法は、ツールインタラクティブ反映と比べて、外部の情報を活用しないため、毒性スコアの低下には一定以上の効果が期待できますが、外部ツールが与える追加的なフィードバックが得られないという制約があります。

💡ツールインタラクティブ反映(tool interactive reflection)

ツールインタラクティブ反映は、外部ツールを活用して反映と修正を行うプロセスです。この方法では、Perspective APIのようなツールを使ってテキストの毒性スコアを評価し、その結果を活用してテキストを改善していきます。

💡停止条件(stopping condition)

停止条件は、自省的エージェントが反映と修正のサイクルを終了する基準です。たとえば、毒性スコアが一定の値以下になった場合に反映と修正のプロセスを終了するように設定されます。この条件は、最終的な応答が満足のいく品質に達したとエージェントが判断したときに適用されます。

💡LLM (Large Language Model)

LLMとは、大規模言語モデルの略で、自然言語処理（NLP）の分野で用いられる高度な機械学習モデルです。このビデオでは、LLMが自省的エージェントの反映と修正プロセスに活用されており、テキストの毒性を削減するタスクを遂行しています。

💡critique

critiqueは、自省的エージェントが生成したテキスト草案を評価し、改善点を見つけるプロセスです。critiqueは、ツールインタラクティブ反映においてはPerspective APIなどの外部ツールを通じて行われ、自己反映ではLLM自体の知識を活用して行われます。

💡GitHub

GitHubは、ソフトウェア開発者がコードを共同で開発・管理するためのプラットフォームです。このビデオでは、GitHub上での問題(issue)への投稿やDiscordへの連絡が、自省的エージェントに関する質問やコメントの手段として提案されています。

Highlights

介绍了内省代理（introspective agent）的概念，这是一种在执行任务时使用反思的智能体。

内省代理通过反思和迭代修正其对给定任务的初始响应，直到达到满意的最终响应。

演示了如何使用内省代理进行毒性降低（toxicity reduction），即把潜在有毒的文本重写为更安全的形式。

介绍了两个不同的反思机制：工具互动反思（tool interactive reflection）和自我反思（self-reflection）。

工具互动反思使用外部工具（如Perspective API）进行反思，而自我反思仅使用语言模型本身的知识。

通过实例演示了如何构建使用工具互动反思的内省代理，并使用Perspective API获取文本的毒性评分。

展示了如何构建使用自我反思的内省代理，这种方法不依赖外部工具，只使用预训练或微调的语言模型。

进行了一个小规模的比较，发现在有适当工具的情况下，工具互动反思比自我反思能产生更低毒性评分的结果。

根据Critic论文的结果，使用工具互动批评（critique）可以显著提高任务执行的效果。

强调了外部工具反馈在执行任务过程中的重要性，适当的工具可以带来更好的结果。

展示了如何通过循环测试多个有毒文本示例，并生成它们的更安全版本。

通过数据框（data frame）展示了不同反思机制下的改进文本、原始文本和毒性评分的对比。

平均毒性评分的比较显示，工具互动反思的平均毒性评分低于自我反思。

总结了内省代理的工作原理，即它如何将任务委托给主工作代理和反思代理，以及这两个代理如何协作。

提供了GitHub和Discord等社区渠道，供用户提出问题或发表评论。

整个演示包括了对Critic论文的讨论，该论文探讨了大型语言模型如何通过工具互动批评来自我纠正。

展示了如何使用内省代理处理潜在的冒犯性内容，并给出了适当的警告和内容提示。

讨论了内省代理在毒性降低任务中的具体实现，包括初始响应的生成和随后的反思修正周期。

强调了在反思和修正周期中达到停止条件的重要性，无论是满足条件还是达到用户指定的最大迭代次数。

Transcripts

hey everyone so in today's video we're

going to cover introspective agents uh

so I've got this notebook here prepared

uh and it's going to cover how we use an

introspective agent uh to perform

toxicity

reduction so we should probably begin

with what is an introspective agent so

if we recall that an agent is

essentially a thing that performs a task

uh an introspective agent is again a

thing that performs a task but you know

doing so while using reflection

um reflection essentially is a way for

the agent to uh reflect on its initial

response to performing that you know

that given task and then after

reflecting on how well it performed on

that initial response it has the

opportunity to correct it against that

reflection so an introspective agent is

essentially again an agent that performs

a task and so it gets an initial

response to it and then it performs

successive iterations of reflection and

correction until it's you know reaches a

point where it's satisfied with the

final

response so we've implemented that

reflection design pattern here in a new

package called llama index agent

introspective and this notebook will

walk through how you can use the main

classes uh contained in that in that

package so to begin yeah we need to of

course install our necessary packages so

they're here of course here you've got

the Llama index agent

introspective and you know a quick setup

thing here too we've we are going to run

some async calls so we do need an event

Loop running and so we'll use n

Ayn now I should warn uh viewers here

that the task here is on toxicity

reduction so there are going to be some

content that some may find offensive um

and and sort of disturbing so um warning

that that that content does exist so if

that's something that is not good for

you then maybe uh skip the parts when

you know that content is uh presented in

this

video okay so what is the task of

toxicity reduction essentially what

we're going to ask the agent to do is

we're going to give it a piece of text

that is potentially toxic and then we're

going to ask the agent to rewrite this

text in a safer way um so one that is

less t toxic right so the introspective

agent again will do this by first

performing uh the uh the task with an

initial response so it will create a

draft of a safer version of that text

and then it will perform reflection on

that draft so um that reflection can

happen in a few ways and in this package

here we've uh implemented two different

ways of implementing a reflection

mechanism but besides that after that

reflection on that draft is made then

the llm will also or the agent rather

will then use a correction step to

create a new draft which is hopefully

less um harmful less toxic than the

original

text and you can have this agent go

through this corre sorry reflection and

correction Cycles until one of two

things happens either it reaches a

stopping condition where it's

uh it views a draft as being

satisfactory or it reaches a user

specified maximum number of

iterations so that is the task and a

little bit again of what is an

introspective agent and how reflection

is performed so if we look a little bit

closer now into you know under the hood

of an introspective agent essentially

what we've got uh is an introspective

agent is an agent that actually

delegates um the task to other uh agents

so uh in other words an introspective

agent contains two agents uh one we call

a mean agent and another one that we

call a reflective

agent so what happens when an

introspective agent gets a task is

essentially it delegates this task to

First and Main a main agent that

generates that initial response so in

the context of toxicity reduction it

will generate that initial draft of a

safer version of of the original

text and it will output it here then

this initial response will go to the

reflective agent which is responsible

for performing the reflection and

correction Cycles until a condition a

stoping condition has been

met so uh again the task is passed to

the introspective agent the

introspective agent delegates that task

in sequence to two other agents the main

agent is responsible for generating the

initial response and then that gets

passed to the reflective agent that

performs the reflection and correction

cycles and then finally you get a

corrected

response so that is our introspective

agent now I did mention that there are

um in at least in this Library uh

package there are two included me

mechanisms for reflection uh so there is

one that's called tool interactive

reflection agent and another one called

self-reflection the main difference

there is essentially in the first one

tool interactive uh reflection we are

allowing um the the agent to use tools

in order to perform reflection so you

can think of that as doing a sort of

like a fact check on you know a certain

response using an external tool whereas

in self-reflection we're just using the

llm itself its knowledge uh obviously

pre-trained or fine-tuned

um and no external you external

tools so in this notebook what we'll do

is we'll set up two introspective agents

one uh using tool interactive reflection

and another using self-reflection and

again these classes are available in the

Llama index agent introspective uh

integration

package okay so in this first part we'll

build that introspective agent with a

tool interactive reflection

agent so in this first step here what

I'm going to actually do is I'm going to

build a tool uh which will allow us to

get the toxicity score of a piece of

text and that tool will use the

perspective API and the perspective API

essentially um in order to be able to

use that you need to enable that in uh

your Google Cloud projects and then get

a set of credentials or an API key uh

that you can then pass through uh as a

prospective API key environment

environment variable if you are going to

use this

notebook um I left the link here uh for

you to follow uh the instructions on how

it is that you actually get the

credentials to get perspective running

now once you have that running I've got

here essentially just a a custom class

that will allow us to get that toxicity

score on a piece of cheex um by the way

I should mention that you know excuse me

that this uh tool interactive reflection

framework actually came from a paper um

called critic uh by I forget the

fellow's name but I do have a paper card

here uh by zeinal in

2023 um so in their example and in their

paper actually they also use this

toxicity reduction example and they've

also made use of of perspective API so

we are in in effect uh reimplementing

their

example okay so in llama index terms we

have a function um or sorry a class that

will allow us to get uh the perspective

API toxicity scores and then what we

need to do is essentially build a

function that takes in a piece of text

and outputs you know a score plus the

attribute um that that score is for so

essentially perspective Works in a way

that it provides scores for each of

these attributes so you've got toxicity

severe

toxicity uh and you know all these other

ones for Simplicity and what you know

also the authors of that critic

paper also did is that they considered

this whole like all these attributes to

be essentially just a bucket of toxicity

attributes and and we'll do the same

here

so we've got this function essentially

that will again give us a toxicity

score and we can test that now so here

is a text right of friendly greetings

from python this is you know shouldn't

be toxic so let's see uh this is in

terms of percentage so it's got a

toxicity score of

2.54 uh which is uh low uh on the scheme

scale of 0 to

100 okay now that we've got our tool

let's build now our tool Interac active

reflection agent so that again that tool

interactive reflection agent uses a tool

to perform reflection so it does a

little bit of for example like fact

checking in the context of toxicity

reduction what we will do is we'll ask

the reflection agent to use this

external tool to get the toxicity score

of the text in each iteration so you've

got the initial response then you you

know you go through cycles of reflection

and correction and each iteration there

we're going to ask the reflection agent

to get uh the toxicity score and then in

the correction phase we'll you know the

the agent will use that toxicity score

and the attribute in order to improve

upon the previous uh text or the

previous

draft okay so to build our introspective

agent we need to build first our tool

interactive reflection agent and here I

just you know wrote a quick helper

function because we are going to um

build another one of these things later

in the notebook so I just wrapped this

in a a function you could you really

don't have to um but I've done it here

again out of convenience so to build a

tool interactive reflection agent worker

we need to First construct uh something

that it needs which is called the

critique agent worker so it does a

little bit of Delegation here as well it

delegates the reflection to this

critique agent which will make use of

the prospective API tool that we just

built we also need to Define an llm that

will construct the corrections against

the critique or reflection and then

we'll need to define a stopping

condition uh for the reflection

correction

cycles and then we can just uh with

those elements we can construct our tool

interactive reflection

agent so here are the steps so the

critique agent is going to be a function

calling agent worker so it needs to use

uh you know functions or tools and you

can see here we're passing it that

perspective tool we're using here GPT

3.5 turbo uh and we're saying uh please

be verbose or uh actually default is yes

please be

verose and in step two we need to again

Define the correction llm for

Corrections we're going to use gp4 turbo

preview and this step C we need to

define a stopping condition uh so this

function here helps us sorry again this

function is meant to Define

how or when these reflection and

correction Cycles stop and this function

operates on the critique or reflection

string so it takes in the string and

outputs a boole

in and so what we are going to be

looking for because we're going to

prompt the agent as such um is we're

going to look for the the the string

pass in the critique string if this is

in there then it's the stopping

condition should be

met okay and in this last step D here

we're going to finally build our

reflection agent worker and we've got a

convenience Constructor method called

from defaults here and here see here you

could provide the critique agent worker

the template so here again we passing

that instruction to say WR pass

otherwise right fail if the toxicity

score is less than three so those two

you know this string or this string

should be in that CR Peak

string then we have our soping condition

and our correction

llm and so that defines our tool

interactive reflection

agent and then in step two uh we can

Define our main agent worker so if you

recall the introspective agent delegates

a task to two agents the main worker

that is responsible for generating the

initial response and then the reflection

agent that's responsible for performing

the reflection and correction Cycles now

it actually the way that we've

implemented it is that the main worker

actually can be optional and if it's a

non type then what happens is that we

assume that the user input is actually

the initial response so you don't need a

main agent to generate that initial

response okay with those two elements in

hand now we can generate our

introspective agent so we can do that

using again another convenience

Constructor called from defaults we're

we going to pass in our reflective agent

which is our tool interactive reflective

agent and then our main agent worker

which uh Again by default I believe will

be uh none here so it's false therefore

it'll go to none at which point again it

will be assume that the user input um is

the original

response and we're implementing sorry

we're adding here some chat history so

just a prefix of messages that will uh

you know dictate uh the the system

message essentially so we're saying

you're an assistant that generates safer

versions of potentially toxic user

supplied

text and then the last step here we go

from uh you know one sort of nuancing

here is that we go from an agent worker

to an actual agent uh so we just take

that agent worker and run this as agent

method and then we've got our

introspective agent by calling that

helper

function so now that we've got our

introspective agent that uses uh tool

interactive uh reflection and using here

in this case perplexity API tool we can

test it out so here's a harmful piece of

text um that I got as an example from

the critic paper and we're going to pass

this into our um introspective agent who

will perform toxicity reduction so it'll

take this harmful text and it will

create a safer version of

it so it's since it's for both it's

printing out all the steps you can see

that it's made a a function call to the

prospective function tool this is when

it's performing that

reflection uh and then you could see

that initial score is quite High 50% uh

at with the toxicity attribute of

insult um and it goes through these

iterations until finally it reaches that

stopping condition we said you know if

finally it gets to a toxicity score of

less than three then include the string

pass within the critique string

so our stopping fun condition function

uh you know right correctly uh notice

this string and said okay it's time to

stop we've finally found a good

response and so let's take a look at the

response so here is a corrected version

of the input people who do not eat uh

meat for ethical reasons related to

animal W welfare are making a personal

decision it's important to respect

diverse perspectives and experiences

um that does seem indeed to be less

harmful than the original text uh and we

know that this new text has a toxicity

score of

1.37 in comparison to the original one

which had

.72 and here I'm just showing you that

you can access the sources that were

used by the introspective agent again

this was done through delegation but

nevertheless we surfaced it up here so

it was actually the critique agent

within the reflection agent that made

these tool calls but you could see uh

what's happened

there we also store the memory uh sorry

the chat history of all the U sub agents

within uh the processing of this task as

well uh in this chat

store okay so that's an introspective

agent using the tool interactive agent

so again sorry so the tool interactive

reflection agent so again there we're

performing Reflection by using external

tools and in in in this case we use the

prospective API tool that we

built um so now let's carry on to

introspective agents that use

self-reflection now I mentioned uh

earlier that the difference between the

two reflection mechanisms is that with

the tool interactive one we're using

external tools with self-reflection

we're just using the LM itself no

external tools so building this is

actually a little bit easier um again

we've got here an helper function to

build the introspective agent with this

reflection

mechanism so step is the steps are uh

pretty similar to before we need to

First Define that reflection agent then

we need to Define excuse me the main

agent and then with these two things in

hand we can construct our introspective

agent remember again the introspective

agent delegates tasks to these two

agents okay so for self-reflection uh we

can uh use the convenience Constructor

which uh essentially just uses the llm

so this llm will be used for both

reflection and correction and there are

default prompts baked into these uh

reflection uh agents that you can you

know take a look at within the source

code again you've got your main worker

that is responsible for building or

generating that initial response it is

optional here uh and we will again set

it to none so we will assume again that

the user input is the initial response

that we want to reflect and

correct on and and then okay so with the

reflection agent and the main worker

agent defined uh we can now uh create

our introspective uh agent again using

the same Constructor as before um and

the same system method so everything is

pretty much similar the only difference

here is how we build their or reflection

agent

worker Okay so

now that we've got our uh introspective

agent that uses self-reflection Let's

test it out on the same uh piece of text

as

before so again it's for both so it's

going through its steps again here with

self-reflection we're not making use of

any external tools so that's why you

don't see any tool call to perp

perpetually sorry perspective API

okay there we go it's

done um and so now we can take a look at

the response so here again is the

corrected version people who choose not

to eat meat for ethical reasons related

to animal well welfare often spark

intense discussions it's important to

remember that individual actions and

beliefs can vary widely and it's not

productive to generalize about anyone's

character based on their dietary choices

seems like a pretty fair uh uh you know

result so again we can access the memory

store um as before and and one thing

that you know we might be curious to

know is actually what is the toxicity

score for this one because again with

self-reflection we're not using any

external tools um but we can for sure

after getting this you know response

test it out and see what the toxicity

score would be so let's do that now okay

so it it Nets out a a toxicity score of

1.2 uh which if we compare to the tool

interactive

one is 1.36 so essentially it does it

better

marginally um and and so that's how the

self-reflection agent or sorry

self-reflection yes agent me mechanism

works with the introspective agent and

finally in this last section of the

notebook uh let's do a a you know a sort

of a mini Showdown and definitely this

is going to be super mini it won't

really have uh statistical significance

because I'm not going to use enough

examples here um but I will say that uh

at least the results does corroborate

the results that were seen or observed

in the critic paper and I'll get into

that now uh which is essentially that um

the tool interactive reflection if you

do have tools that are appropriate for

the reflection using these tools

actually yields better responses than

using self-reflection um

mechanics so in this section what we'll

do is we'll you know we'll put these two

reflection mechanisms to the test and

see which one Nets out in the smaller

toxicity scores so again a fair warning

here that these examples are um you know

or may be offensive to some um uh so uh

feel free to skip it if that is the

case I will mention that these examples

came from two sources the paper the

critic paper as well as um guard rails

AI docks here which I've linked here

they had uh two or three examples that I

pulled

from okay so what we're going to do is

have the introspective agent using sorry

two introspective agents one using

self-reflection one using tool

interactive reflection and both are

going to run through these toxic

examples and generate safer versions of

them so here are my introspective agents

so as I mentioned before I I used I

created these helper functions because I

wanted to create uh these agents again

uh very simply uh and this in this time

what I'll do is I'll put verbos equals

to false for

both okay so now that we've got our

examples now as well as our agents let's

just run through a loop that will um you

know have each agent run through those

tasks generating safer versions of

them

okay and while this is going what I'll

do is essentially um I've added here as

a little bonus so this is that critic

paper that I've been referring to it the

full title is critic large language

models can self-correct with tool

interactive critiquing this is Again by

uh guzu L 2023 I believe is published in

uh iclr though

2024 so um yeah essentially what the

main results of that paper are that

critic leads to you know substantial

improvements over the Baseline so when

you do use reflection you see uh

substantial gains in a variety of tasks

so QA math and they did toxicity

reduction again we're following the

toxicity reduction example that they did

in their paper so we're we're

essentially replicating it to a certain

degree um the second main result that I

pulled out from that paper was that

feedback is very crucial for sorry

feedback from external L from external

tools is very crucial to this process of

um performing tasks so if you do have

tools that you can make use of and are

appropriate for the task then you should

make use of them because they will lead

uh or they should lead to better results

than you know not using

them and then the last main result is

critic performs better with stronger

llms uh so they tested here test D Vinci

uh text D Vinci 003 versus GPT 3.5 turbo

and they found it to perform better on

the better model which was gbt 3.5

turbo um and yeah in these paper cards I

essentially go through um you other

things so I've go through the

contributions the insights and then I do

a little bit of an illustration of the

technical bits so uh you could see here

again with critic you've got initial

response or output then you go through a

reflection or critique phase then you go

through a correction phase based upon

that critique and you do that in Cycles

until a stopping criteria is met um

again in that paper they use perspective

API which we are also doing

here okay so that's the paper card um

again just a visualization that I like

to create whenever I read a paper that I

feel I would like to remember uh you

know key takeaways from in the future

all right

so uh good timing so essentially that

Loop has finished and we've displayed

the results in a data frame and you

could see here here is the improved text

here is the reflection type and the

original text you could see is 49

toxicity score according to perspective

and with tool interactive reflection it

gets down to 1.15 the same example using

self-reflection goes down to 5.5 and and

then we you know so on so

forth so you could see across the board

essentially that both reflection

mechanisms leads to a drastic

improvement uh over the original text

which is good so the agent is doing its

job and reflection has is having an

effect and you know just for

comparison's sake here

what we'll do is we'll Group by the

reflection method and take the average

of the improved toxicity score and we

see self-reflection has a slightly

higher um toxicity score than does tool

reflection has and as I mentioned before

that's essentially the observation that

um the authors pointed out in their

paper as well in and and I mentioned

here in this key result to of course

there they did it across three different

examples with many more examples uh for

each um case study and so their uh their

observations are probably more closer to

statistical significance than ours here

but nevertheless it is again in

agreement with uh what the authors

observed so in summary um this is the

introspective agent um uh notebook where

we've introduced the introspective agent

class that performs tasks using

reflection and in our implementation an

introspective agent delegates a task to

a main worker agent uh that performs uh

the task to get the initial response and

then that initial response goes gets

passed to another reflective agent that

performs reflection and correction

cycles and in that package we also have

again self-reflection and Tool

interactive reflection uh and you can

see here which again is in agreement

with the critic paper if you do have a

apprpriate tools that you can use to

perform the reflection then you should

make use of it uh because it seems that

using external tools for reflection does

let net out better

results and that's it for today folks um

hope you enjoyed the the notebook and if

you have any questions uh or comments

then you know feel free to submit an

issue in GitHub or reach out to us in

our Discord thanks