What's next for AI agentic workflows ft. Andrew Ng of AI Fund

Summary

The transcript discusses the evolution of AI agents and their impact on computer science, highlighting the significance of iterative, agentic workflows in enhancing AI performance. It emphasizes the potential of design patterns like reflection, multi-agent collaboration, planning, and debate in boosting productivity and achieving remarkable results. The speaker also underscores the importance of adapting to these agentic workflows and the role of fast token generation in facilitating iterative processes.

Takeaways

• 🧠 The importance of neural networks and GPUs in AI development, with Andrew Ng's significant contributions through his work on Coursera, deeplearning.ai, and Google Brain.
• 📝 The contrast between non-agentic and agentic workflows in AI, where the latter involves iterative processes similar to human thought and revision.
• 🤖 The concept of AI agents and their potential to transform AI applications, emphasizing the shift from traditional AI usage to more interactive and collaborative models.
• 🔄 The iterative process of agentic workflows that involve planning, execution, revision, and testing, leading to improved outcomes over non-agentic approaches.
• 📈 The case study highlighting the effectiveness of agentic workflows with GPT-3.5, where incorporating agentic strategies improved performance over zero-shot prompting.
• 💡 The four broad design patterns observed in AI agents: reflection, multi-agent collaboration, planning, and multi-agent debate, each with varying degrees of maturity and reliability.
• 🔍 The use of self-reflection in AI coding agents to identify and correct errors in their own generated code, demonstrating a level of autonomy and self-awareness.
• 👥 The potential of multi-agent systems, where different AI agents can take on various roles and collaborate effectively, leading to complex problem-solving and decision-making.
• 🚀 The anticipation of rapid advancements in AI capabilities due to agentic workflows, suggesting a significant shift in how AI applications are designed and utilized.
• 🕒 The need for patience and dedication when working with AI agents, as the iterative process may require more time for the AI to deliver optimized results.
• 💬 The closing thoughts on the journey towards AGI (Artificial General Intelligence) and how agentic reasoning design patterns could contribute to this long-term goal.

Q & A

What is the main focus of the discussion in the transcript?

The main focus of the discussion is the development and application of AI agents using various design patterns, particularly in the context of neural networks and large language models (LMs).

Who is Andrew and what is his notable contribution to the field of AI?

Andrew is a renowned computer science professor at Stanford, known for his early work in developing neural networks with GPUs. He is also the creator of Coursera, popular courses like deeplearning.ai, and the founder and early lead of Google Brain.

What are the two different workflows for using LMs as mentioned in the transcript?

The two workflows mentioned are non-agentic and agentic. The non-agentic workflow involves typing a prompt and generating an answer, similar to asking a person to write an essay without using backspace. The agentic workflow is more iterative, involving multiple interactions with the LM, such as writing an outline, conducting web research, drafting, revising, and iterating until the desired outcome is achieved.

How does the agentic workflow improve results compared to the non-agentic workflow?

The agentic workflow delivers remarkably better results as it allows for a more iterative and reflective process. This includes the ability for the LM to self-evaluate its own code, revise it based on feedback, and continue to improve through several rounds of thinking and revising.

What is the significance of the study using the human eval benchmark?

The study using the human eval benchmark demonstrated that an agentic workflow with GPT-3.5 outperformed even GPT-4 in certain coding tasks. This highlights the effectiveness of the agentic approach and its potential to enhance the performance of AI systems.

What are the four broad design patterns mentioned in the transcript?

The four broad design patterns mentioned are reflection, planning, multi-agent collaboration, and two-use (using LMs for various tasks like analysis, information gathering, and action).

How does self-reflection work in the context of an LM coding agent?

Self-reflection involves prompting the LM to review and evaluate the code it generated, identify any issues, and suggest improvements. This process can lead to the LM creating a better version of the code based on its own feedback.

What is the concept of multi-agent collaboration?

Multi-agent collaboration involves using multiple LMs, each prompted to act in different roles (e.g., coder, critic, CEO, designer), to work together on a task. These agents can have extended conversations and collaborate to achieve complex outcomes, such as developing a software program.

Why is fast token generation important in agentic workflows?

Fast token generation is crucial in agentic workflows because it allows for quicker iterations. The LM can generate tokens at a pace much faster than a human can read, which facilitates the rapid exchange and refinement of ideas, leading to more efficient and potentially higher-quality outcomes.

What is the significance of the trend towards AGI (Artificial General Intelligence) as mentioned in the transcript?

The path towards AGI is viewed as a journey rather than a destination. The use of agentic workflows and design patterns is seen as a way to make progress towards achieving AGI, with the potential to improve AI systems' ability to perform a wide range of tasks autonomously and effectively.

What is the advice given for effectively using AI agents in one's workflow?

The advice given is to be patient and allow AI agents time to process and respond, even if it takes minutes or hours. Just like delegating tasks to a team member and checking in later, it's important to give AI agents the time they need to provide the best possible outcomes.

Outlines

00:00

🤖 Introduction to AI Agents and Their Impact

The speaker begins by acknowledging Andreu's contributions to computer science, particularly in the realm of neural networks and AI. He introduces the concept of AI agents, emphasizing their potential as a significant trend in AI development. The speaker shares his experience with problem sets from a Stanford course, highlighting the importance of iterative learning and refinement. He contrasts non-agentic workflows, where tasks are completed in one go without revision, to agentic workflows that involve multiple iterations and refinements, leading to better outcomes. The speaker also discusses the effectiveness of using agentic workflows with GPT-3.5 and the potential for improved performance over newer models like GPT-4.

05:01

📚 Design Patterns in AI Agents

The speaker delves into the design patterns observed in AI agents, noting the chaotic yet promising landscape of AI research and development. He outlines four key design patterns: reflection, multi-agent collaboration, planning, and multi-agent debate. Reflection involves self-assessment and improvement of code, while multi-agent collaboration introduces the concept of having different agents perform specialized tasks. Planning algorithms are highlighted for their ability to adapt and overcome failures, and multi-agent debate showcases the power of diverse perspectives leading to enhanced outcomes. The speaker emphasizes the importance of these patterns in boosting productivity and the potential for AI to expand its capabilities.

10:04

🚀 Future Trends and the Path to AGI

In the final paragraph, the speaker discusses the future trends in AI, predicting a significant expansion of AI capabilities due to agentic workflows. He challenges the conventional expectation of immediate responses from AI, advocating for patience and the understanding that AI agents may require more time to deliver high-quality outcomes. The speaker also emphasizes the importance of fast token generation for iterative processes in agentic workflows. He expresses excitement for upcoming AI models and believes that these advancements could bring us closer to achieving AGI, viewing it as a journey rather than a destination. The speaker concludes with a hopeful outlook on the role of agent workflows in propelling AI forward.

Keywords

💡Neural Networks with GPUs

Neural Networks with GPUs refer to the use of graphical processing units to accelerate the training and deployment of artificial neural networks, which are computational models inspired by the human brain and are a key component of deep learning. In the context of the video, this technology was pivotal in the early development of AI and machine learning, as it enabled faster and more efficient processing of large datasets, leading to significant advancements in the field.

💡Coursera

Coursera is an online learning platform that offers a wide range of courses, including those related to computer science and AI, from top universities and institutions worldwide. It has played a significant role in democratizing education by making high-quality learning resources accessible to a global audience. In the video, Coursera is mentioned as one of the platforms where Andrew has contributed to the dissemination of knowledge in AI and deep learning.

💡Deeplearning.ai

Deeplearning.ai is a website founded by Andrew Ng that provides educational content and resources focused on deep learning, a subset of machine learning that involves the use of neural networks with many layers to learn representations of data. The platform is known for its high-quality courses and materials that cater to both beginners and experienced professionals in the field of AI. In the context of the video, it is one of the initiatives through which Andrew has contributed to the AI community.

💡Google Brain

Google Brain is a research project by Google that focuses on advancing the field of deep learning and artificial intelligence. It involves the collaboration of researchers and engineers working on various AI-related problems, from improving the performance of neural networks to developing new machine learning algorithms. In the video, Andrew's role as a founder and early lead of Google Brain is highlighted, emphasizing his significant contributions to the company's AI initiatives.

💡AI Agents

AI Agents refer to autonomous systems or software entities that can perceive their environment, make decisions, and take actions to achieve specific goals. In the context of the video, AI agents are discussed as a transformative trend in AI development, where the focus shifts from non-agentic workflows to more interactive and iterative processes that mimic human problem-solving strategies.

💡Iterative Workflow

An iterative workflow is a process of repeated cycles of development and refinement, where the output of each cycle is reviewed, revised, and improved upon. This approach is contrasted with a non-iterative, one-shot process in the video, where tasks like writing an essay or coding are completed in a single attempt without revision. Iterative workflows are emphasized as more effective in achieving higher quality results, as they allow for continuous improvement and refinement.

💡Human Evaluation Benchmark

The Human Evaluation Benchmark is a tool or a set of standards used to assess and compare the performance of AI systems or algorithms against human performance in specific tasks. It provides a quantitative measure of how well an AI model can accomplish a given task and is often used to evaluate the effectiveness of new AI technologies or improvements to existing ones. In the video, the speaker uses the Human Evaluation Benchmark to demonstrate the improved performance of AI agents in an iterative workflow compared to non-agentic approaches.

💡Planning Algorithms

Planning algorithms are computational methods used in AI to determine a sequence of actions that a system should take to achieve a specific goal or set of goals. These algorithms are crucial in creating AI agents capable of autonomous decision-making and problem-solving, as they simulate and evaluate different possible courses of action to select the most effective one. In the video, planning algorithms are discussed as a key component in enabling AI agents to handle complex tasks and recover from failures.

💡Multi-Agent Collaboration

Multi-Agent Collaboration refers to the interaction and cooperation of multiple AI agents to achieve a common goal. This approach leverages the strengths of different AI systems or models to solve complex problems that may be beyond the capabilities of a single agent. In the context of the video, multi-agent collaboration is presented as an emerging and powerful design pattern that can lead to surprising and effective outcomes, such as collaboratively developing a complex program or game.

💡Agentic Reasoning

Agentic Reasoning refers to the ability of AI systems to simulate human-like thought processes and decision-making, including planning, problem-solving, and learning from experience. It involves creating AI agents that can interact with their environment in a purposeful and goal-oriented manner. In the video, the speaker emphasizes the importance of agentic reasoning as a design pattern that can significantly boost productivity and lead to more advanced AI capabilities.

💡Fast Token Generation

Fast Token Generation is the ability of AI models, particularly language models, to quickly produce a large number of tokens or text elements during the iterative process of agentic workflows. This speed is crucial for maintaining the efficiency of the workflow, as it allows for more iterations and refinements within a shorter period. In the context of the video, fast token generation is highlighted as an important trend that can enhance the performance of AI agents, even when using slightly lower quality models.

Highlights

Andreu's early contributions to neural networks with GPUs and his role in creating Coursera and deeplearning.ai.

The importance of iterative processes in AI workflows, compared to non-agentic, one-shot methods.

The surprising effectiveness of agentic workflows in improving AI performance, even surpassing newer models like GPT-4.

The concept of reflection as a powerful tool in AI design patterns, allowing systems to self-evaluate and improve their output.

The potential of multi-agent collaboration, where different AI agents can work together, each playing different roles.

The use of two-use systems in expanding the capabilities of language models, especially in areas like computer vision.

The impact of planning algorithms on AI's ability to autonomously handle failures and reroute processes.

The increasing role of AI agents in personal workflows, such as research, where they can assist in gathering and analyzing information.

The significance of fast token generation in agentic workflows, allowing for quicker iterations and potentially better results.

The anticipation of future AI advancements due to agentic reasoning design patterns.

The need for patience and dedication when working with AI agents, as they may require more time to process and respond effectively.

The potential of lower quality models with faster token generation to outperform slightly higher quality models with slower token generation.

The ongoing journey towards AGI and how agent workflows might contribute to this progression.

The excitement around upcoming AI models like Cloud T5, CL 4, GPT-5, and Gemini 2.0.

Transcripts

all of you uh know Andreu in as a famous

uh computer science professor at

Stanford was really early on in the

development of neural networks with gpus

of course a creator of corsera and

popular courses like

deeplearning.ai also the founder and

Creator uh and early lead of Google

brain uh but one thing I've always

wanted to ask you before I hand it over

Andrew while you're on stage uh is a

question I think would be relevant to

the whole audience 10 years ago on

problem set number two of cs229 you gave

me a

b and I was wondering I looked it over I

was wondering what you saw that I did

incorrectly so anyway Andrew thank you

Hansen um looking forward to sharing

with all of you what I'm seeing with AI

agents which I think is the exciting

Trend that I think everyone building in

AI should pay attention to and then also

excited about all all the other uh on

Sak presentations so hey agents you know

today the way most of us use Lish models

is like this with a non- agentic

workflow where you type a prompt and

generates an answer and that's a bit

like if you ask a person to write an

essay on a topic and I say please sit

down to the keyboard and just type the

essay from start to finish without ever

using backspace um and despite how hard

thises is L's do it remarkably well in

contrast with an agentic workflow this

is what it may look like have an AI have

an LM say write an essay outline do you

need to do any web research if so let's

do that then write the first draft and

then read your own first draft and think

about what parts need revision and then

revise your draft and you go on and on

and so this workflow is much more

iterative where you may have the L do

some thinking um and then revise this

article and then do some more thinking

and iterate this through a number of

times and what not many people

appreciate is this delivers remarkably

better results um I've actually been

really surprised myself working these

agent workflows how well how well they

work I's do one case study at my team

analyzed some data uh using a coding

Benchmark called the human eval

Benchmark released by open a few years

ago um but this says coding problems

like given the nonent list of integers

return the sum of all the all elements

are an even positions and it turns out

the answer is you code snipper like that

so today lot of us will use zero shot

prompting meaning we tell the AI write

the code and have it run on the first

spot like who codes like that no human

codes like that just type out the code

and run it maybe you do I can't do that

um so it turns out that if you use GPT

3.5 uh zero shot prompting it gets it

48% right uh gp4 way better 607 7% right

but if you take an agentic workflow and

wrap it around GPT 3.5 I say it actually

does better than even

gbd4 um and if you were to wrap this

type of workflow around gb4 you know it

it it also um does very well and you

notice that gbd 3.5 with an agentic

workflow actually outperforms

gp4 um and I think this has and this

means that this has signant consequences

fighting how we all approach building

applications so agents is the ter of

around a lot there's a lot of consultant

reports talk about agents the future of

AI blah blah blah I want to be a bit

concrete and share of you um the broad

design patterns I'm seeing in agents

it's a very messy chaotic space tons of

research tons of Open Source there's a

lot going on but I try to categorize um

bit more concretely what's going on

agents reflection is a tool that I think

many of us should just use it just works

uh to use I think it's more widely

appreciated but actually works pretty

well I think of these as pretty robust

technology when I use them I can you

know almost always get them to work well

um planning and multi-agent

collaboration I think is more emerging

when I use them sometimes my mind is

blown for how well they work but at

least at this moment in time I don't

feel like I can always get them to work

Rel Lively so let me walk through these

four design patterns in the few slides

and if some of you go back and yourself

will ask your engineers to use these I

think you get a productivity boost quite

quickly so reflection here's an example

let's say ask a system please write code

for me for a given task then we have a

coder agent just an LM that you prompt

to write code to say you def du task

write a function like that um an example

of

self-reflection would be if you then

prompt the LM with something like this

here's code intended for a toas and just

give it back the exact same code that

they just generated and then say check

the code carefully for correctness sound

efficiency good construction CRI just

write prompt like that it turns out the

same l that you prompted to write the

code may be able to spot problems like

this bug in line Five May fix it by blah

blah blah and if you now take his own

feedback and give it to it and reprompt

it it may come up with a version two of

the code that could well work better

than the first version not guaranteed

but it works you know often enough for

this be wor trying for a lot of

applications um to foreshadow to use if

you let it run unit test if it fails a

unit test then he why do you fail the

unit test have that conversation and be

able to figure out fail the unit test so

you should try changing something and

come up with V3 by the way for those of

you that want to learn more about these

Technologies I'm very excited about them

for each of the four sections I have a

little recommended reading section at

the bottom that you know hopefully gives

more references and again just the

foreshadow multi-agent systems I've

described as a single coder agent that

you prompt to have it you know have this

conversation with itself um one Natural

Evolution of this idea is instead of a

single code agent you can can have two

agents where one is a coder agent and

the second is a Critic agent and these

could be the same base LM model but that

you prompt in different ways where you

say one your expert coder right code the

other one say your expert code review to

review this code and this Tye of

workflow is actually pretty easy to

implement I think it's such a very

general purpose technology for a lot of

workflows this would give you a

significant boost in in the performance

of LMS um the second design pattern is

to use many of where already have seen

you know LM based systems uh uh using

tools on the left is a screenshot from

um co-pilot on the right is something

that I kind of extracted from uh gp4 but

you know LM today if you ask it what's

the best coffee maker web search for

some problems um will generate code and

run code um and it turns out that there

are a lot of different tools that many

different people are using for analysis

for gathering information for taking

action for personal productivity

um it turns out a lot of the early work

in two use turned out to be in the

computer vision Community because before

large language models lm's you know they

couldn't do anything with images so the

only option was that the LM generate a

function called that could manipulate an

image like generate an image or do

object detection or whatever so if you

actually look at literature it's been

interesting how much of the work um in

two years seems like it originated from

Vision because LMS would blind to images

before you know gp4 and and and lava and

so on um so that's two use and it

expands what an LM can do um and then

planning you know for those of you that

have not yet played a lot with planning

algorithms I I feel like a lot of people

talk about the chat GPT moment where

you're wow never seen anything like this

I think if not used planning alums many

people will have a kind of a AI agent

wow I couldn't imagine the AI agent

doing this I've run live demos where

something failed and the AI agent

rerouted around the failures I've

actually had quite a few of those moment

wow you can't believe my AI system just

did that autonomously but um one example

that I adapted from a hugging GPT paper

you know you say this general image

where the girls read where a girl is

reading a book and it posts the same as

a boy in the image example. jpack and

please subscribe the new image for your

voice so give an example like this um

today we have ai agents who can kind of

decide first thing I need to do is

determine the post of the boy

um then you know find the right model

maybe on hugging face to extract the

post then next need to find a post image

model to synthesize a picture of a of a

girl of as following the instructions

then use image to text to and then

finally use text of speech and today we

actually have agents that I don't want

to say they work reliably you know

they're kind of finicky they don't

always work but when it works is

actually pretty amazing but with agentic

loops sometimes you can recover from

earlier failures as well so I find

myself already using research agents for

some of my work where one of piece of

research but I don't feel like you know

Googling myself and spend a long time I

should send to the research agent come

back in a few minutes and see what it's

come up with and and it sometimes works

sometimes doesn't right but that's

already a part of my personal

workflow the final design pattern multi-

Asian collaboration this is one of those

funny things but uh um it works much

better than you might think

uh uh but on the left is a screenshot

from a paper called um chat Dev uh which

is completely open which actually open

source many of you saw the you know

flashy social media announcements of

demo of a Devon uh uh Chad Dev is open

source it runs on my laptop and what

Chad Dev doeses is example of a

multi-agent system where you prompt one

LM to sometimes act like the CEO of a

software engine company sometimes Act

designer sometime a product manager

sometimes I a tester and this flock of

agents that you built by prompting an LM

to tell them you're now Co you're now

software engineer they collaborate have

an extended conversation so that if you

tell it please develop a game develop a

GOI game they'll actually spend you know

a few minutes writing code testing it uh

iterating and then generate a like

surprisingly complex programs doesn't

always work I've used it sometimes it

doesn't work sometimes it's amazing but

this technology is really um getting

better and and just one of design

pattern it turns out that multi-agent

debate where you have different agents

you know for example could be have ch

GPT and Gemini debate each other that

actually results in better performance

as well so having multiple simulated air

agents work together has been a powerful

design pattern as well um so just to

summarize I think these are the these

are the the the uh patterns of seen and

I think that if we were to um use these

uh uh patterns you know in our work a

lot of us can get a prity boost quite

quickly and I think that um agentic

reasoning design patterns are going to

be important uh this is my small slide I

expect that the set of T AI could do

will expand dramatically this year uh

because of agentic workflows and one

thing that it's actually difficult

people to get used to is when we prompt

an LM we want to response right away

um in fact a decade ago when I was you

know having discussions around at at at

Google on um it called a big box search

we type a long prompt one of the reasons

you know I failed to push successfully

for that was because when you do a web

search you one of responds back in half

a second right that's just human nature

we like that instant grab instant

feedback but for a lot of the agent

workflows um I think we'll need to learn

to dedicate the toss and AI agent and

patiently wait minutes maybe even hours

uh to for a response but just like I've

seen a lot of novice managers delegate

something to someone and then check in 5

minutes later right and that's not

productive um I think we need to it be

difficult we need to do that with some

of our AI agents as well I saw I heard

some loss um and then one other

important Trend fast token generation is

important because with these agented

workflows we're iterating over and over

so the LM is generating tokens for the

elm to read so be able to generate

tokens way faster than any human to read

is fantastic and I think that um

generating more tokens really quickly

from even a slightly lower quality LM

might give good results compared to

slower tokens from a better LM maybe

it's a little bit controversial because

it may let you go around this Loop a lot

more times kind of like the results I

showed with gbd3 and an agent

architecture on the first slide um and

cand I'm really looking forward to Cloud

5 and uh CL 4 and gb5 and Gemini 2.0 and

all these other wonderful models that

may are building

and part of me feels like if you're

looking forward to running your thing on

gp5 zero shot you know you mayble to get

closer to that level performance on some

applications than you might think with

agenting reasoning um but on an early

model I think I I I I think this is an

important Trend uh uh and honestly the

path to AGI feels like a journey rather

than a destination but I think this typ

of agent workflows could help us take a

small step forward on this very long

journey thank

[Applause]

you