LangChain Agents: A Simple, Fast-Paced Guide

Summary

This video delves into the concept of agents and Lang chain, exploring how they combine large language models with practical tools to create powerful virtual assistants. It discusses the thought-action-observation pattern agents follow, the role of prompts and output parsers, and how agents utilize tools to complete tasks. The video also covers different types of agents, such as react doc store and self-ask with search, and introduces new frameworks like plan and execute for handling complex use cases. With examples and code snippets hosted on GitHub, viewers are guided through the process of implementing these agents, highlighting their potential in automating tasks and decision-making.

Takeaways

• 🧠 Large language models (LLMs) are capable of reasoning and acting in external worlds, which can be combined to create 'agents' that can think and perform tasks.
• 🛠️ Agents follow a 'thought-action-observation' pattern, where they think about what to do next, perform actions using tools, and observe the output to inform their next thought.
• 📝 Agents use prompts to instruct the LLM to format its response according to a specified output schema, which includes tools available to the agent and chat history.
• 🔧 The agent's output parser translates the LLM's output into a Python object, which can be an 'agent action' with tools and inputs or an 'agent finish' with results.
• 🔄 The agent executor runs the tool with the input received from the thought step, abstracting away the thought-action-observation loop from the user.
• 🏢 The script discusses various types of agents, including 'react' agents that follow the same loop and 'self-ask with search' agents that ask themselves sub-questions to find answers.
• 📈 The 'plan and execute' agent framework was created to address the need for stability and complexity in enterprise use cases, separating planning and acting for more reliability.
• 💡 Baby AGI and the 'plan and solve' paper inspired the creation of autonomous agents that can devise plans and maintain memory between model calls to achieve objectives.
• 🎉 The 'generative agent' framework allows characters to plan, interact, form relationships, and celebrate events in a simulated universe, showcasing the potential for agent-based AI interactions.
• 🛑 Autonomous agents can be expensive to run due to the number of calls they make and the risk of infinite loops, making them more of fun projects than practical solutions currently.
• 🔗 Tools and toolkits in LangChain can be explored and created using specific functions and decorators, allowing for structured and efficient agent interactions.

Q & A

What is the concept of 'Lang Chain' and how does it relate to large language models?

Lang Chain is a framework that combines large language models with agents to perform tasks. It allows agents to reason and act in external worlds, utilizing the reasoning capabilities of large language models to think through actions and execute them using provided tools.

How does an agent in Lang Chain understand the instructions given to it?

An agent in Lang Chain understands instructions through prompts and output parsers. The prompt instructs the language model to format its response according to a specified output schema, which includes a list of tools the agent can use, chat history, and examples. The output parser then translates the language model's output into a Python object that the agent can act upon.

What is the 'thought-action-observation' pattern mentioned in the script?

The 'thought-action-observation' pattern is a cycle followed by agents in Lang Chain. It starts with a thought, where the agent decides what to do next using the language model. Then it moves to the action step, where the agent executes the decided task using available tools. Finally, it observes the outcome, which informs the next thought process.

Can you explain the role of 'agent executor' in the Lang Chain framework?

The agent executor in Lang Chain is responsible for running the tool with the input received from the thought step. It abstracts away the thought-action-observation loop from the user, providing an interface for interaction and executing the agent's decisions.

What are some of the early stopping conditions that can be applied in the Lang Chain framework?

Early stopping conditions in Lang Chain can include a limit on the number of iterations or a time limit that has passed. These conditions prevent the agent from running indefinitely and ensure that a final answer is reached within a reasonable timeframe.

How is the 'react doc store' agent different from other agents in Lang Chain?

The 'react doc store' agent uses the React framework to search and lookup documents within a document store. This is different from the more commonly used Vector store abstraction and allows for efficient document retrieval.

What is the 'self-ask with search' agent and how does it work?

The 'self-ask with search' agent is capable of asking itself a series of sub-questions to arrive at an answer. For example, if asked for the hometown of the reigning men's U.S. Open champion, it would first find out who the champion is, then ask where he is from, and use these intermediate answers to provide the final response.

What are the benefits and drawbacks of using the 'plan and execute' agent framework?

The 'plan and execute' framework offers reliability by separating planning and acting concerns, which can also allow for the use of smaller, cheaper, fine-tuned models for specific tasks. However, it requires more language model calls, which can increase latency for user-facing applications.

Can you provide an example of how the 'plan and execute' agent framework is used?

An example of using the 'plan and execute' framework is creating and sending a sales report to an email using data only available in an SQL database. The agent plans the steps needed, such as querying the database and composing the email, then executes each step to achieve the final goal.

What is the 'generative agent' framework and how does it differ from other agent frameworks?

The 'generative agent' framework is inspired by the idea of creating a simulated universe where characters plan, interact, and form relationships. It differs from other frameworks by focusing on the generation of content and interactions within a simulated environment rather than task execution.

How can one create custom tools and toolkits in Lang Chain?

Custom tools can be created using the 'tool_from_function_factory' method, which requires a function that accepts a string input and returns a string output, along with a name and description. Toolkits are arrays of related tools, and can be defined using a decorator to structure tool classes for multi-argument tools.

Outlines

00:00

🧠 Introduction to Agents and Lang Chain

This paragraph introduces the concept of agents and Lang Chain, a framework that combines large language models with the ability to act in external worlds. It explains how agents can be used to create specialized virtual assistants with enhanced capabilities. The video promises a deep dive into the history and evolution of agents, starting with their inspiration from the React Miracle and Self-Ask with Search paper. It also mentions the ongoing development of agents with new features, such as OpenAI's function calls. The paragraph outlines the 'thought-action-observation' pattern that agents follow, which involves thinking about the next steps, taking actions, and making observations based on the output. It also discusses the role of prompts and output parsers in enabling agents to understand and respond to tasks. The video concludes this section with an example of how to use agents in code, highlighting the simplicity of implementing agents with just a few lines of code.

05:02

🛠 Exploring Advanced Agent Frameworks and Tools

The second paragraph delves into the complexities of advanced agent frameworks and tools within Lang Chain. It begins by discussing the 'plan and execute' framework, which separates planning and acting to increase reliability and allows for the use of smaller, fine-tuned models. The paragraph provides a practical example of using this framework to create and send a sales report via email, utilizing data from an SQL database. The video also touches on the inspirations behind this framework, including 'Baby AGI' and the 'Plan and Solve' paper. It then introduces the concept of 'generative agents' inspired by the Generative Agent paper, which creates a simulated universe where characters interact and form relationships. The paragraph concludes with a discussion on tools and toolkits, explaining how to discover available tools, create new ones, and the importance of understanding prompts and output parsers for effective agent interaction.

Keywords

💡Large Language Models (LLMs)

Large Language Models (LLMs) are advanced AI systems designed to process and understand human language on a massive scale. They are capable of reasoning and generating human-like text based on the input they receive. In the context of the video, LLMs are the core component that enables agents to think and reason through tasks. The script mentions that agents use LLMs to decide what actions to take next, highlighting the importance of LLMs in creating autonomous and intelligent agents.

💡Agents

In the video, 'agents' refer to AI entities that combine the reasoning capabilities of large language models with the ability to interact with external tools and environments. Agents are designed to perform tasks by thinking through the steps and then executing those steps using available tools. The script describes agents as a fusion of LLMs and external action capabilities, creating a 'virtual assistant on steroids,' which can be programmed to complete complex tasks autonomously.

💡Thought-Action-Observation Pattern

The Thought-Action-Observation pattern is a sequence that agents follow to complete tasks. It begins with the 'thought' phase, where the agent uses the LLM to decide on the next course of action. 'Action' is the phase where the agent executes the planned task using available tools. Finally, 'observation' involves the agent observing the results of its actions to inform its next thought. The video script explains this pattern as the fundamental process that agents use to interact with the world and complete tasks.

💡Prompts and Output Parsers

Prompts and output parsers are essential for the communication between the agent and the LLM. A 'prompt' is a set of instructions given to the LLM to format its response according to a specified schema, which includes information about the tools the agent can use. An 'output parser' then takes the LLM's response and translates it into a Python object that the agent can understand and act upon. The script illustrates how these components work together to enable the agent to understand and execute tasks as directed by the LLM.

💡Agent Executor

The 'agent executor' is the component responsible for running the tools with the inputs provided by the agent's thought process. It is the part of the agent that directly interacts with external tools and systems to perform actions. The video script explains that the agent executor abstracts away the thought-action-observation loop from the user, providing a seamless interface for task execution.

💡React Miracle

The 'React Miracle' seems to be a term used in the video to refer to the influence or inspiration behind the development of agents and their capabilities. While the script does not provide a detailed definition, it suggests that the React Miracle had a significant impact on the evolution of agents, possibly in terms of their ability to reason and act autonomously.

💡Auto GPT and Baby AGI

Auto GPT and Baby AGI are projects mentioned in the script that are related to the development of autonomous agents. Auto GPT is a framework for creating AI agents that can perform tasks autonomously, while Baby AGI is described as an early experiment in laying down the architecture for an AGI (Artificial General Intelligence) system using available tools. The script positions these projects as influential in the conceptualization and development of the agents discussed in the video.

💡Open AI's Function Calls

Open AI's function calls are a feature that allows agents to execute specific functions or tasks. The script mentions this feature as an expansion to the capabilities of agents, suggesting that it enables them to perform a wider range of actions. While the script does not provide a detailed explanation, it implies that this feature is part of the ongoing evolution and enhancement of agent capabilities.

💡Plan and Execute Autonomous Agent

The 'plan and execute autonomous agent' is a framework introduced in the video to address the need for more complex use cases and stability in agent operations. This framework separates the planning and execution phases, with a 'planner' that uses a language model to reason and plan multiple steps ahead, and an 'executor' that carries out those plans. The script explains that this approach enhances reliability and allows for the use of smaller, more task-specific language models.

💡Generative Agent

The 'generative agent' is a concept from the script that refers to a type of agent that can generate content or take actions in a simulated environment. The video describes a universe where these agents plan, interact, form relationships, and even celebrate events, suggesting a level of autonomy and complexity beyond simple task execution. The generative agent represents an advanced form of AI that can simulate complex behaviors and interactions.

Highlights

Agents in LangChain combine internal reasoning of large language models with external tool use, creating powerful virtual assistants.

Agents follow a thought-action-observation pattern, enabling them to perform complex tasks by iterating through this loop.

Prompts and output parsers are crucial for agents, guiding the large language model to produce actionable outputs.

LangChain supports multiple action agents, including those that use the REACT framework for document search and look-up.

Self-ask with search agents decompose complex questions into simpler sub-questions, enhancing their ability to find accurate answers.

LangChain's agents can handle various tools, store chat history, and parse outputs into Python objects.

The Plan and Execute agent framework separates planning from execution, improving reliability and allowing for the use of optimized models.

Autonomous agents, such as those inspired by Baby AGI, can autonomously generate and prioritize tasks to achieve given objectives.

Generative agents simulate environments where characters interact, form relationships, and perform daily activities, showcasing potential future applications.

Toolkits in LangChain package related tools together, such as the Gmail toolkit, to streamline agent capabilities.

LangChain supports creating custom tools using the tool from function factory method, enabling flexible input and output configurations.

The LangChain repo provides examples and source code to help users understand prompts and how agents function.

LangChain's integration with OpenAI's function calls expands the capabilities of agents, pointing to future advancements in the field.

Verbose mode in LangChain allows users to see the agent's thought process, enhancing transparency and debugging.

Plan and Execute agents address complexities and stability issues in enterprise use cases by separating planning from action execution.

Transcripts

agents blank chain allows us to ship

them like hotcakes but what are they

large language models can reason via

internal thoughts like explaining jokes

reasoning through a math problem but

they've also shown the ability to act in

external worlds as seen in the seiken

and the web GPT papers so what if you

brought the two together you get an

agent an agent has access to tools to do

things in a large language model to

think you give them a task and it will

think of what it should do to complete

the task and then do it using the tools

we've given it it's like creating your

own specialized virtual assistant on

steroids today I will do a deep dive on

agents and Lang chain how they were

first influenced by the react miracle

and self-ask with search paper and how

it has been continuously involving with

Concepts like autonomous agents commonly

known through projects like Auto GPT and

baby AGI and now expanding with new

features like open ai's function calls

throughout this video we'll go through

code examples to show how agents work as

always the code will be hosted on GitHub

if you'd like to see more content like

this don't forget to like and subscribe

now let's dive in most of the action

agents implemented in line chain follow

the thought action observation pattern

where it first starts with a thought

this is where the agent thinks of what

he needs to do next using the llm but

how does the agent understand what the

alarm tells it so if you've noticed on

the previous slide we can see that

agents also have prompts and output

parsers The Prompt instructs the llm to

format its response according to a

specified output schema it also contains

a list of tools the agent has access to

and could have a chat history and some

examples the output schema specified

enables the agent's output parser the

parse the large language models output

into a python object in this case the

parsed object will either be an instance

of agent action having as instance

variables the tools to use and its input

or agent finish having as instance

variables the results to send back now

let's get back to our diagram we now

know that the thoughts step can either

result in continuing or stopping if we

get an agent finished we're done

otherwise we got an agent action and we

move on the action step as users the

agent's thought action observation Loop

is abstracted away from us and what

we're really interacting with is the

agent executor during the action step

it's the agent executor that will run

the tool with the tool input we have

received from the thought step the agent

class does not directly run the tool by

running the tool this leads us to an

observation which is basically the

output of the tool which will be used to

generate our next thought the agent

executor goes through this thought

action observation Loop repeatedly until

we arrive at a final answer or until we

hit our early stopping conditions which

could either be the number of iterations

or time that passed did you know that

all of this is possible in less than 10

lines of code let's check it out after

making our Imports let's instantiate an

llm and loader tools we can initialize

our agents by passing in the tools large

language model and the agent type and

that's it now let's ask the agent to

find the average price for a one bedroom

apartment in New York City and calculate

a 20 deposit on it know that we're able

to see the agent's thought process

thanks to using the verbose keyword

argument Now link chain supports

multiple action agents let's quickly go

over them before starting note that

everything that has react and its name

will go through the same thought action

observation Loop we've just talked about

let's start with the react doc store

which uses the react framework to easily

search and look up documents within a

doc store as opposed to the more

commonly used Vector store abstraction

up next we have self ask with the search

let's explain this with an example by

asking the hometown the reigning man's

U.S open Champion the agent first asked

who's the reigning man champion and gets

the name as an intermediate answer then

it asks where he's from to then receive

his hometown as an intermediate answer

which essentially answers the final

question the agent basically asked

itself a bunch of sub questions and

answered them until it came up with a

response now all the other ones ending

with react description are similar but

with a few subtle differences here's the

diagram illustrating it the chat agents

are compatible within your chat messages

API needed to use GPT 3.5 and gpt4 the

conference conversational agents store

the conversation in memory to pass the

context over the future iterations out

of all of them only one supports using

tools that accepts multiple inputs there

are other Frameworks of agents that you

should know about which are currently in

the experimental folder of the like

chain repo like plan and execute

autonomous and generative agent let's

take a closer look at the planet execute

agent first off it came more out of a

necessity action agents worked well

until a couple of pain points emerged

people started using agents for more

complex use cases and as it became more

adopted in Enterprise a need for

stability also became more apparent line

and execute agents where the response to

this growing pain let's check back the

control flow for Action agent we see

that after sending our request the agent

might then look for a tool runs the tool

and then examines the output of the tool

and so on and so forth another way to

solve the request to handle more

complexity is the first plan I had all

the steps to take and then execute each

step this framework requires two things

a planner and an Executor the planner

will have a language model that will be

utilized as a reason and plan multiple

steps ahead the executor can use the

same language model or another one the

advantages are that there is now more

reliability since we're separating the

planning and acting concern this also

enables us to swap the executor's LM for

smaller and cheaper fine-tuned models

optimized for specific tasks a

disadvantage is that we're making more

language model calls so for user-facing

application model latency is something

to keep in mind now let's jump into the

code we'll use plan and solve to create

and send a sales report to our email

using data only available in an sqldb if

you want the code check out the GitHub

repo now the actual code is very simple

after looting our environment variables

we instantiate our large language model

and we set up our Gmail toolkit and our

DB toolkit these toolkits are just

multiple tools packed together we put

all the tools together inside an array

loader planner using gpt4 and are

executor using the same model but notice

that we could have used something else

here when instantiator plan an execute

agent and then ask it to essentially

make a sales report summing up the sales

of each salesman and sending an email to

the head of cells it will take some time

but you can see the whole chain of

reasoning which is really fascinating to

look at and of course the end result is

in our inbox if you're curious about

what inspired this framework check out

baby AGI and the plan and solve paper

which were cited as major Inspirations

from the Lang chain Team all the links

will be down in the description below

talking about baby ATI it was one of the

first few autonomous agents it started

as a semi-serious semi-fun experiment to

lay down with an AGI architecture could

look like with current available tools

so the cool thing about it is that you

give it an objective for example make a

thousand dollars then in step one the

first task it pulls from the database is

to make a list on how to earn a thousand

dollars the execution agent in this case

will devis a plan that could look like

one open a bank account to make digital

painting three sell it on the

marketplace Step 2 involves a context

agent that stores output from Step 1

along with metadata in a vector database

the system maintains some form of memory

between model calls step 3 examines the

results from Step One and create new

tasks and re-prioritize them according

to the objective and this essentially

Loops back until it gets done as you can

see these kind of autonomous agents are

super expensive to run because of the

number of calls they can make and they

could also run into Infinite Loops for

now autonomous agents are mainly fun

projects another interesting agent

framework is called generative agent

from the generative agent paper the team

has created a sim-like universe where

characters planned and went about their

days interacted with each other form

relationships and even celebrated

birthdays together does that sound

familiar now going over to tools you can

find out available tools and toolkits by

running the following command toolkits

are simply a couple of related tools

packed together in an array for example

we can see in the source code that the

Gmail toolkit contains tools related to

using Gmail you can create a tool by

using the tool from function Factory

method it will need a function which

accepts a string input and returns a

string output a name and a description

using pedentic there are also ways to

provide more information about inputs

there's also a decorator that you can

use to Define Tools in a structured tool

class if you want to create your own

multi-argument tools now for prompts and

output parsers to actually get a good

understanding of how they work check out

my previous video link chain 101 the

complete beginner's guide in any case I

suggest taking a look at the source code

of the line chain repo and to go through

the prompts written you will see for a

given Asian framework what the prompt is

and that will help you build a deeper

intuition on how the magic happens as

this video is getting a bit long I'll

cover openai function calls in another

video it's pretty exciting to see the

field move so fast and this might be the

future direction for agents don't forget

to subscribe if you want to be notified

and as always thank you for watching and

have a wonderful day