why you suck at prompt engineering (and how to fix it)

Summary

The video script offers an in-depth guide on prompt engineering with AI language models, emphasizing the importance of understanding the underlying science to maximize their potential. It humorously compares different levels of prompt engineering skills to a spectrum, highlighting the 'midwit' trap where people overcomplicate tasks. The speaker shares his journey and the strategies he uses to build efficient AI systems, including the use of specific techniques like role prompting, chain of thought prompting, emotion prompting, and few-shot prompting. The script also discusses the economic aspect of AI solutions, advising on the choice of model based on cost and performance. It concludes with a formula for creating effective prompts and its application across various AI use cases, such as AI agents, voice assistants, and automations, underlining the significance of prompt engineering in the burgeoning AI industry.

Takeaways

• 🧠 **Understanding Prompt Engineering**: The ability to effectively prompt AI models is crucial for building AI systems and getting value from language models like GPT.
• 🚀 **Role Prompting**: Assigning an advantageous role to the AI model and enriching the role with key qualities can significantly increase the accuracy of prompts.
• 🤖 **Task Specificity**: Clearly defining the task with a verb and being as descriptive as possible allows the AI to understand exactly what is required.
• 💡 **Chain of Thought Prompting**: Providing step-by-step instructions for the AI to follow can dramatically improve the accuracy of complex tasks.
• 📈 **Performance Increase**: Techniques like role prompting, chain of thought, and emotion prompting can lead to significant performance improvements in AI systems.
• 💌 **Emotion Prompt**: Adding emotional stimuli to prompts can enhance performance, truthfulness, and informativeness of the AI's output.
• 📚 **Contextual Information**: Giving the AI context about the environment it's operating in can help improve performance by making the task more relatable.
• 📉 **Few Shot Prompting**: Providing a few examples (3-5) can greatly increase the accuracy and performance of the AI without needing fine-tuning.
• 📝 **Notes and Tweaks**: Using a notes section to remind the AI of key aspects and to add final details can fine-tune the output without restructuring the entire prompt.
• 🔑 **Markdown Formatting**: Structuring prompts with markdown can improve readability and potentially the AI's understanding and performance.
• 💡 **Positive Reinforcement**: Encouraging the AI model and using positive feedback can lead to better responses and higher quality outputs.

Q & A

What is the main issue the video aims to address?

The video addresses the issue of ineffective prompt engineering in AI systems, explaining why many people struggle with it, and how they can improve their skills to build better AI systems.

What is the 'midwit' concept mentioned in the video?

The 'midwit' concept refers to individuals who overcomplicate tasks, making them more difficult and inefficient. In the context of the video, a midwit is someone stuck in the middle of the IQ spectrum, not leveraging simple solutions like low IQ people, nor understanding the advanced techniques like high IQ people.

What is the significance of the 'Prompt Engineering' in building AI systems?

Prompt engineering is crucial as it directly impacts one's ability to extract value from AI models. It involves crafting instructions that AI systems can follow to perform specific tasks, and mastering this skill is essential for creating efficient and cost-effective AI solutions.

Why is the video's presenter moving away from fancy video production?

The presenter is focusing more on his business and team, building software and educating his community. He prefers to spend less time on video production and more on these activities that are directly related to his business goals.

What is the role of English in programming AI models?

English serves as a new programming language for AI models. By writing effective prompts in English, one can instruct AI models to perform tasks, which can replace the need for large blocks of code, making AI more accessible and efficient.

What is the difference between conversational and single shot prompting?

Conversational prompting is interactive, allowing for follow-up prompts and adjustments by a human operator, which is good for personal use. Single shot prompting, on the other hand, is a one-time instruction that is integrated into a system for automation, requiring no human intervention and is ideal for scalable AI systems.

How can a well-written prompt replace hundreds of lines of code?

A well-written prompt can encapsulate the logic and instructions that would otherwise require extensive coding. By providing clear and specific directions, an AI model can execute complex tasks based on the prompt, eliminating the need for the same task's code.

What are the components of the 'perfect prompt formula'?

The components of the perfect prompt formula are role, task specifics, context, examples, and notes. Each component is backed by scientific research or prompting techniques that enhance the accuracy and performance of the AI model.

Why is markdown formatting used in prompt engineering?

Markdown formatting is used to structure prompts for better readability and to help the AI model understand the structure of the prompt better. It uses headings, bold text, lists, and other formatting tools to organize the information logically.

What is the 'Lost in the Middle' effect and how does it apply to prompt engineering?

The 'Lost in the Middle' effect refers to the phenomenon where language models perform best when relevant information is placed at the beginning or the end of a context, and performance significantly worsens when critical information is in the middle. In prompt engineering, this understanding helps in structuring prompts effectively.

How can one optimize the cost and performance of an AI system?

One can optimize the cost and performance of an AI system by mastering prompt engineering to utilize cheaper and faster models effectively. By crafting precise and effective prompts, one can reduce the reliance on more expensive models and achieve similar results at a lower cost and higher speed.

Outlines

00:00

😀 Introduction to Prompt Engineering

The speaker begins by addressing the audience's potential lack of skill in prompt engineering and promises to explain why they may be stuck in a 'midwit' range. The video aims to elevate the audience's understanding of prompt engineering, moving them from a plateau to a level of expertise. The speaker uses a meme to illustrate the convergence of low and high IQ individuals on similar solutions, contrasting this with the overcomplication by 'midwits.' The goal is to transition from relying on templates to understanding the science behind prompt engineering, which is crucial for building AI systems.

05:00

🚀 Building Reliable AI Systems with Single Shot Prompting

The speaker differentiates between conversational and single shot prompting, emphasizing the importance of the latter for creating scalable and reliable AI systems. They argue that while conversational prompting might improve one's job performance, single shot prompting allows for the development of valuable AI systems. The speaker also highlights the significance of English as a programming language for AI, stating that effective prompts can replace extensive code.

10:01

🤔 Prompt Engineering Misconceptions and the Importance of Understanding

The speaker discusses common misconceptions about prompt engineering, noting that many believe they are proficient when they are actually only good at conversational prompting. The video aims to correct this by teaching the audience the fundamentals of prompt engineering, which is essential for creating AI voice systems, AI agents, and custom AI tools. The speaker also shares personal insights about their business and the motivation behind teaching these skills.

15:02

📈 The Perfect Prompt Formula for AI Systems

The speaker outlines the components of an effective prompt, which includes role, task specifics, context, examples, and notes. Each component is backed by research or a discovered technique, and the speaker provides a detailed explanation of how to apply these components to build better AI systems. The goal is to help the audience understand the science behind prompt engineering to create more accurate and efficient AI models.

20:02

📉 The Diminishing Returns of Adding Examples to Prompts

The speaker discusses the research results related to the effectiveness of adding examples to prompts, noting that accuracy increases significantly with each additional example up to a certain point. They suggest that providing 3 to 5 examples is sufficient for most tasks, as more examples increase the cost and complexity of the prompt without substantially improving performance.

25:02

🔑 Final Touches: Notes Section and Markdown Formatting

The speaker introduces the final part of the prompt, the notes section, which serves as a reminder for key aspects of the task and a place to add final details. They also discuss the importance of markdown formatting for structuring prompts, making them more readable and understandable for the AI model. The speaker emphasizes the effectiveness of positive reinforcement when interacting with AI models and the potential benefits of using persona-based prompts.

30:03

🎓 Conclusion: Mastering Prompt Engineering for AI Success

The speaker concludes by emphasizing the importance of mastering prompt engineering to succeed in the AI space. They provide a comprehensive guide that can be applied to various AI systems, including AI agents, voice agents, and AI automations. The speaker also discusses the business implications of prompt engineering, suggesting that those who do not master these skills will be outcompeted by those who do.

Keywords

💡Prompt Engineering

Prompt engineering refers to the skill of effectively instructing AI models, such as language models, to perform specific tasks. In the video, it is emphasized as a critical ability for leveraging AI systems to their full potential. The script discusses the importance of understanding the science behind prompt engineering to move beyond basic usage and into building sophisticated AI applications.

💡Midwit

The term 'midwit' is used in the script to describe someone who overcomplicates tasks, often due to a lack of understanding of the underlying principles. It is part of a meme referenced in the video, contrasting with 'low IQ' and 'high IQ' individuals who coincidentally reach similar solutions. In the context of the video, avoiding 'midwit' behavior is crucial for efficient prompt engineering.

💡Conversational Prompting

Conversational prompting is a method of interacting with AI where the user has a back-and-forth dialogue with the AI model, allowing for follow-up prompts and modifications. The video explains that while this method is good for personal use and tweaking responses, it is not ideal for building automated systems that require consistent and error-free outputs.

💡Single Shot Prompting

Single shot prompting is a technique where the AI model is given a prompt and expected to produce a correct response in one go, without the opportunity for follow-up prompts. This method is highlighted in the video as essential for creating automated, scalable AI systems that can be reliably integrated into business processes.

💡Role Prompting

Role prompting involves assigning an AI model a specific role or identity, which can enhance its performance by up to 25% as mentioned in the script. By defining a role that is advantageous to the task at hand, the AI is better guided to perform the task with higher accuracy. This technique is part of the broader strategy to improve the effectiveness of prompts.

💡Chain of Thought Prompting

Chain of thought prompting is a technique where the AI is instructed to think through a problem step by step, either by providing step-by-step instructions or by encouraging it to articulate its reasoning process. The video notes that this method can significantly boost accuracy, especially for complex problems, by making the AI's problem-solving process more transparent.

💡Emotion Prompt

An emotion prompt is a short phrase containing emotional stimuli, used to enhance the performance of a prompt. The script mentions that adding emotional stimuli like 'this is very important to my career' can increase the accuracy of simple tasks by 8% and complex tasks by 115%. This technique leverages the AI's ability to respond to emotional cues to improve its output.

💡Few Shot Prompting

Few shot prompting involves providing the AI with a few examples of inputs and desired outputs to guide its learning. The video references research showing that providing just a few examples can massively increase the performance of a prompt, with most gains achieved between 10 to 32 well-crafted examples. This technique is used to teach the AI the expected response format without extensive fine-tuning.

💡Lost in the Middle Effect

The 'lost in the middle' effect is a phenomenon where information presented in the middle of a long context is less likely to be remembered or acted upon by the AI model. The video suggests structuring prompts to place critical information at the beginning or end to improve performance. This understanding helps in crafting prompts that are more likely to be effective.

💡Markdown Formatting

Markdown formatting is a way to structure text using plain text conventions. In the context of the video, it is used to organize and structure prompts for better readability and to potentially enhance the AI's understanding of the prompt's structure. The video suggests that using markdown headings can help in clearly delineating the different components of a prompt, such as role, task, and examples.

💡AI Systems

AI systems in the video refer to the various applications and tools powered by artificial intelligence that can be built using prompt engineering skills. These systems can range from email classification tools to AI agents and voice assistants. The script emphasizes that mastering prompt engineering enables the creation of valuable AI systems that can significantly impact businesses and the AI space.

Highlights

The video discusses the concept of prompt engineering and its importance in AI systems, emphasizing that improper prompt engineering can lead to suboptimal results.

The speaker introduces a meme comparing low IQ, midwit, and high IQ individuals to illustrate the complexity of problem-solving, relating it to prompt engineering.

The video emphasizes the importance of understanding the science behind prompt engineering to avoid being stuck in the 'midwit' range.

The speaker explains how prompt engineering skills can significantly impact the value one can extract from AI language models.

The concept of 'conversational' versus 'single shot' prompting is introduced, with the latter being more suitable for AI systems and automation.

The speaker shares personal experiences and challenges in prompt engineering, admitting to being part of the problem and offering solutions.

The video highlights the importance of crafting well-written prompts that can replace hundreds of lines of code, emphasizing the potential of AI in programming.

The speaker discusses the role of English as a 'programming language' in the context of AI, where effective prompts can replace traditional coding.

The video presents a 'perfect prompt formula' for building AI systems, which includes role, task, specifics, context, examples, and notes.

The speaker explains the significance of role prompting, where assigning an advantageous role to the AI can increase the accuracy of prompts.

The concept of 'chain of thought' prompting is introduced, which involves providing step-by-step instructions to the AI for complex tasks.

The video discusses the impact of emotional stimuli in prompts, which can enhance the performance of AI systems.

The speaker emphasizes the importance of providing examples in prompts, which can significantly improve the accuracy and performance of AI systems.

The video introduces the 'lost in the middle' effect, which suggests that information placed at the beginning or end of a prompt is more likely to be followed by the AI.

The speaker discusses the use of markdown formatting in prompts, which can improve readability and potentially the performance of AI systems.

The video concludes with a practical example of how to apply the learned techniques in prompt engineering to improve AI system outcomes.

Transcripts

you probably suck at promt engineering

and in this video I'm going to tell you

why how you can fix it and how you

cannot be the guy in the middle here of

this mid with me so that's might a

little bit off topic but if you give me

a second I'll explain how this applies

to the majority of people who are trying

to do prompt engineering and build AI

systems and why it's probably holding

you back because you're stuck in this

midwit range so if you haven't seen this

meme before basically the low IQ people

and the high IQ people kind of converge

on the same solution uh as you see here

so we have the guy using Apple notes on

one side and the genius using Apple

notes on one side and in the middle you

have the midwit who's over complicating

it making it very difficult and painful

for themselves and then we have the same

thing with NES Cafe Classic on both

sides and in the middle we have the

midwit struggling with all these

different types of coffee and fancy

methods so how does this apply to prompt

engineering I know you're asking

considering you clicked on a video

that's about prompt engineering but it's

actually a not so bow curve when it

comes to prompt engineering and uh on

the far left we have the stupid person

who is just using chat GPT and prompting

it as they wish kind of just throwing

things in there on the far side we have

what we're trying to get you to after

this video which is a genius who has a

toolkit of prompts and understands the

science behind it and in the middle we

have probably you right now which is uh

I mean no no disrespect to these other

YouTubers cuz I've made videos on on

proing myself like I'm I'm I'm part of

the problem here but uh what these are

all about is chat gbt prompt templates

and and sort of taking the thinking away

from you and putting it in the hands of

this template that they've created so uh

I'm not going to sh on my videos too

much uh because these videos were

talking more conceptually as well so I'd

say I'm on that line and the content of

this presentation in this video is

intended to take you from this plateau

of someone trying to do PR engineering

but not actually understanding the the

science behind it which is what we're

going to go into this video point of

this video is to take you from someone

who's on that Plateau as you can see

here um and get you up to the sort of

genius and and very capable PR engineer

who's able to do great things with these

language models and it's so important

because your ability to prompt them and

and provide instruction of these models

directly impacts your ability to get

value out of them so if there's this

amazing new technology called llms and

you're better at using them you're going

to go further in the AI space and

further in life if you can better send

instructions to these models so

continuing on uh you may be wondering

hey why is this new style why is the

camera on the different side why is

everything so casual um and that's

because uh I've been wasting a not

wasting but I've been spending a lot of

time on my videos uh the past while as

you may have noticed some of you people

starting to think that I'm a YouTuber um

and I'm I've never really thought of

myself as a YouTuber personally um I'm a

businessman and YouTube is how I get

clients for my business and I think you

guys are starting to see me as a as a

YouTuber and I I really as much as I

love making videos and teaching you guys

everything what I really like doing is

working on my business and working my

team and building the cool software that

we're building through genive and work

on the morning side and also the cool

stuff we do with my my education

Community as well and teaching them how

to start their own businesses like I so

probably less Fancy videos that require

a lot of time and editing and and if I

have anything interesting to share and I

want to talk about it like in this video

because this video is coming out of me

seeing so many people that I talk to in

my community not understanding this

fundamental skill and it is so

fundamental but people have this

misconception that they know how to De

it which I'm going to break like just

absolutely destroy if you in this video

uh and rebuild your skills as a prompt

engineer so doing this because if I have

something to talk to you about and I

think it's important for you all um then

I'm going to share it and also you may

be wondering why do I do this at all and

it's because I have a SAS and it helps

agency owners to build AI solutions for

businesses so if I don't teach you guys

how to do prom engineering you're never

going to use my SAS so I have to do this

stuff so that I can succeed and and make

all the money with the SAS that I want

so I'm you guys get a byproduct of me

trying to build my SAS which is helping

you to learn these things so anyway

atics so why you're probably bad at

prompt engineering have conversational

prompt engineering versus single shot

conversational is what everyone thinks

is prompt engineering and they go onto

chat GPT and they go hey hey yeah I got

this got this cool prompt template and

they Chuck in there and they can get

some responses from it and they're like

man I'm so good at this and then they

switch off and think that they're a

prompt engineer and they know how to do

this stuff um this of course is human

operated there are follow-up prompts

that you can do so you can say oh could

you please like modify this a little bit

and because of these follow-up prompts

it's very forgiving in terms of what you

can say um and how you can tweak it to

get the RO responses and this really is

just good for personal use if you're

working at a job and you might want to

streamline some of the work play that

you do there great like I mean Chad GPT

is an incredible software and I use it

the time as well so I'm not not

on it but it is conversational prompting

and on the other side is single shot

prompting which is something that we can

actually bake into a system uh that can

be automated and can be part of a sort

of ongoing ongoing system or flow uh

where an AI task is embedded in it um

there are no follow-up prompts because

there's no no human involved in most

cases there's no room for error in that

case you can't have jgpt putting hey

here is the answer and they put in the

answer it just needs to give you the

answer every single time while the

system's going to break uh because of

this because if we can prompt it into

something that is reliable we can

actually have a very scalable system

that is AI built into it which is ideal

for these AI assisted systems and this

is really how you can create value so

the benefit of conversational prompting

skills which many of you will have I'm

sure is that it might make you better at

your job and might make your boss a bit

more money CU you're able to do more

work um maybe make you a bit more money

on the on the process but the benefit of

these single shot systems where we can

build an AI task to do a specific

function every every single time

reliably is that it will allow you to

build AI systems worth potentially

thousands of dollars a piece as as I've

done as many people in my community have

done as well if you don't believe me I

don't care furthermore on this point of

why you should take prompt engineering

seriously Andre Kathy here uh says the

hottest new programming language is

English and this is no dummy he is a

founding member of open AI he's also a

leading AI researcher what he means here

by saying the hottest new programming

language is English is that you being

able to write instructions in English is

going to allow you to one generate code

if you want to so you can translate from

English to codee that's one way of

programming in English technically but

another way is that if you can write

effective prompts you can replace the

the the programming required with a

massive program or a massive script you

can write a prompt that effectively does

all of the things that that that script

would have done so you can replace large

blocks of code with a well-written

prompt now which is really what I want

you guys to focus on and say well I can

have the abilities of a developer if I

can write these prompts well using llms

properly um and furthermore this guy

also this guy Liam otley I've founded a

couple AI companies I have my own AI

agency Morningside AI I have my own AI

education Community uh my tripa

accelerator and I also have a software U

my AI SAS called agentive which is

really what my focuses on right right

now and I've got some pretty smart

people working for me I'm not the brains

of the operation anymore I I hope I was

at one point but my CTO Spencer has like

five six years of NP uh experience and

he does some really cool stuff for us

and a lot of what I'm going to sharing

in this in terms of how you should be

doing your prod engineering and what

I've learned and what I now use is from

him so you might think I'm just some

goofball who's been doing YouTube for 12

months uh but I do have teamed and I've

paid people who are a lot smarter than

me to give me this knowledge so now I'm

giving it to you so now I want you to

remember this a well-written prompt can

replace hundreds of lines of code going

back to what I said before this is I

think it's my quote but I'm just going

to say someone said it cuz someone must

have said it but that's essentially what

you can do if you write a well-written

prompt um now here's an example so

there's there a video that will have

just gone out recently on my channel

where I manage my phone finances with AI

I set up a system where my assistant can

send money I can send screenshots these

things here through the system and out

comes the other side a tracker for all

my expenses within my notion um it

automatically extracts the extracts the

transactions from the screenshots

categorizes them stores them in my

expense data database within the notion

and this is kind of the system here you

can pause and take a look but basically

so it took me 2 hours to write a very

good prompt that can success categorized

format and then pass the data over to

notion um and that's ended up saving 8

hours per month for my system so example

there not the best one but you get the

idea um if you write a good prompt you

can replace what would have taken like

to me for me to do this expensive system

with code would have taken a a whole lot

longer and it would have been extremely

messy um but the AI can just throw all

the information at it say hey look this

is what I want you to do with it and

outcomes the transactions ready to go

into notion and no we're still not ready

to move forward because you need to

understand that if you can just get this

skill right that many people don't have

correct they think they can do

conversational prompt engineering and

that's going to be enough for them to go

in and build these systems but in AI

voice systems which are all the rage

right now I've done a ton of videos on

you can go watch them on my channel AI

voice systems if you can't prompt

correctly if you don't have good prompt

engineering skills you can't do AI voice

systems if you don't have good prompt

engineering skills you can't create AI

agents like gpts if you don't have good

prompt engineering skills you can't

build ai's tasks into AI automations

like on zapia and make Etc and you can't

build custom AI tools on relevance and

stack Ai and these other platform so if

you can't just get this thing right and

watch the rest of this video it's not

going to be a retention hookie and and

for your Tik Tok brain I don't care if

you watch the rest of it but I'm telling

you if you don't take the time to

actually soak in this information I'm

about to tell you and and get good at

this prompt engineering skill you are

not going to make any money in AI

because everything depends on it and

finally what I want to do is a little

comparison of the two different types of

people you can be you can either watch

this video and come out on the right

side here or you can continue to do your

whatever you think you're doing when

you're prompt engineering um and you can

be like the guy so go on the left the

midw he has a handy bag of prompt

templates he gets stuck when something

doesn't work because he doesn't

understand what what the template's even

doing so then he uses a more expensive

and a smarter model like he moves from

3.5 turbo to four turbo and he goes oh

yeah well now it works because he gets

the models to do the work inad of

himself so by doing this he creates

slower and more expensive systems and

therefore he struggles to create systems

that are actually valuable for the

clients cuz if it's costing them a lot

and they're really slow there's less

value for the client right and then

number six he gives up on trying to

start an AI business and get into this

AI solution space and then like some of

you guys in the comments they become a

triaa as a scam goofball and blame it on

the model and not your inability to

learn how to write English and then on

the right we have the guy that you want

to be uh he has a toolkit of prompt

components and methods based on Research

which I'm going to take you through in

this video he approaches problems like

an engineer he skillfully applies these

techniques he achieves the desired

performance with fastest and cheapest

model available so he uses the cheapest

model he can get and uses his skills to

make it do what he needed to do

therefore he's able to create lightning

quick and affordable AI systems for

clients that create actual value because

they're cheap and they're fast and then

therefore he actually makes money

because these clients like wow this

thing is awesome and number seven this

guy then finds other AI Chads like him

who know how to do prompt engineering

and are making money with AI and with

him and his friends they all get AI Rich

um yes I'm selling the dream there but

that is what's possible if you can get

this thing right and that is what myself

and a bunch of the other guys that I was

just namam with they're all doing it uh

it's happening um whether you like it or

not so be like this guy don't be like

this guy um yeah there you go so now we

get into the perfect prompt formula for

building AI systems which is the meat

and poates of this video um Beware Of

The Prompt formula as I mentioned you

don't want to be the guy who relies on

the formula and while is while I am

giving you a formula in this video I've

put it in asx's and user capital letters

so that you understand that I'm kind of

taking the piss out of formulas because

what I'm teaching you in this is going

to be the science behind them um so that

you guys if you run into an issue you'll

understand hey look I can apply this

technique to try and fix it so you'll

actually be able to write good prompts

forever if you understand the stuff I'm

going to teach and you actually absorb

it so components of this prompt are role

task specifics context examples and

notes and behind each of these

components is a related uh scientific

paper or some research that has been

done or some prompting technique that

has been discovered and backed up with a

research paper that you can see on

screen here we have roll prompting Chain

of Thought prompting emotion prompt F

shot prompting and lost in the middle

all of these are going to be covered in

the next section to this video so let's

jump into it um oh before we do that

actually what each of these techniques

have is a increase in accuracy or

performance for props and I'm going to

retention hook you here with all these

question marks because over time we're

going to reveal just how much

performance improvements you can get so

if you stack all of these up together uh

you get an increase in performance on

your PR

um just a lot of these are very easy to

implement um but you're going to get a

massive increase I'm not going to tell

you how much it is but a huge increase

just by applying these simple simple

techniques so we're going to be using an

example for this video which is an email

classification system uh and the the AI

task here in the middle uh is where

we're going to have be sending our

prompt and in this case it's going to be

someone comes onto uh someone's website

they fill out a form that form then gets

sent the form submission gets sent by

email to the company the CEO or the Ops

guy uh to his email and he gets it and

then normally has to read through it and

then classify it and and take action

from there but what we're going to be

doing is imagining a system where there

is this AI task or this AI node and

make.com or whatever you want to use

where the email comes in and then it's

going to be classified using our prompt

into opportunity needs attention or

ignore label so super basic system I

wanted to use as an example here let's

get into it um we're going to be

building up a prompt over time of of how

we can apply this techniques to make the

to make this thing better and perform

better so starting off we have the

typical chat GPT prompt if you asked any

mid midwit well not even midwit this

guy's the stupid guy uh if you asked any

regular uh bottom feeder chat GPT user

they' probably give you a prompt like

classify the following email into ignore

opportunity or need detention labels and

then they' paste in the email right so

this is our starting point this is the

typical CHT prompt and this is as far on

the on the IQ scale on the left as you

can go so we're breaking down by

component we're starting off with the

rooll I know for you Tik Tok brains here

you're probably going to look at this

and be like ah there a lot of writing

but uh can you just pause this video uh

I'm not going to go over all of it I

think some of you already know some of

these components Ro prompting is

something that you've definitely done

before but I want to draw attention here

to the research results with this little

rocket ship to show that it's increasing

the accuracy uh when you assign an

advantageous role in your role prompting

by saying you are an email

classification expert uh trained to be

the assist this it can increase the

accuracy of your prompts and the

performance of them by 10.3% and

secondly if you give complimentary

descriptions of your abilities to

further increase accuracy you can get up

to 15 to 25% increase in total so this

is as simple as here's the example you

are a highly skilled in Creative short

form content script writer that is the

role with a knack for crafting engaging

informative and concise videos so you

add a role and then you give it key

qualities like engaging informative and

concise and you basically hype it up and

tell it man you're so amazing at this

this this so you need have a role that

is strong and tells it that is

advantageous to what it's doing so if

you're solving a math problem you are an

expert math teacher and then you can

give it some more examples after that of

the key quality so takeaways here select

the role that is advantageous for the

specific task EG math teacher for math

problems and then enrich the rooll I

like that word enrich the rooll with

with additional words to highlight how

good it is at that task super simple um

that's Ro prompting so this is what

we're going to be doing to kind of tie

everything together in this video which

is a before and after so this was the

this was the low IQ one remember this so

this is our starting point and here we

have what happens after we add in the

roll thing so you're going to need to

pause this as this thing gets bigger

it's kind of hard for me to put the

whole prompt on the screen uh but the

before and after um you're going to have

the r prompt here highlighted and well

low lighted in Black uh so you can see

what we've changed so here we've still

got the task here we've still got the

bit before but it's just now part of a

Li and pront we have the role included

as well you are an experienced email

classification system that accurately

categorizes emails based on the content

and potential business

bagged great so task now going back

there that's pretty helpful this is

actually the task um so the thing that

most people actually put into ches or

into the prompt is the task itself so

it's basically just telling it what it's

going to do uh usually starting with a

verb we want to say generate a this

Analyze This write this but be

descriptive as possible while also

keeping it brief so an example here is

generate engaging and Casual Outreach

messages for users looking to promote

their services in the dental industry

especially focusing on the integration

of AI tools to scale businesses your

messages should be direct so it's

telling it what it should do use a verb

nothing too crazy here um but what I

will mention is that this is where

because we're doing these single shot

systems we need to insert values cuz

it's going to have our prompt written

and then we need to be throwing

different like in this case the email

content is the variable that we need to

put in this place so in this case you

see that I have the dental industry as

the niche and the pink one here which

the integration of tools as the offer um

this is from an earlier video that I've

done within the task is where you can

insert the variables that are going to

be used uh throughout the system so if

you go back a little bit uh we have the

email content variable and you can see

here that it's already become part of

the task so classify the D here's the

variable based input that we want then

we have the technique that's associated

with the task component um and that is

Chain of Thought prompting this is

something that's fairly common now and

pretty widely known um it involves

telling the model to think step by step

without our instructions or B yet you

can provide it with step-by-step

instructions uh for it to work through

each time which is my kind of preferred

way of doing it so here's the example um

we take this script writer example as

well um and in this case if you just

give it a list of six points so hook the

viewer in briefly explain provide one

two F standing facts described so we're

giving it step-by-step instructions on

how it should perform the task and the

research results of of thought prompting

being incorporated into your prompts

it's a 10% accuracy boost on simple

problems I me that's like very very

simple problems like solve this or 4

plus 2 equals blah BL blah uh but 90%

accuracy on complex multi-state problems

which is likely what many of you are

going to be uh dealing with with the

system that you're trying to build so

90% accuracy boost is pretty insane and

uh considering you only have to write up

a little list of what it should do chain

of th promting something you should uh

you should really incorporate uh key

takeaway here the more complex the

problem the more dramatic the

Improvement using chain of Thor

prompting so that's the task if we go

across now you see that we've included a

chain of Thor component to the task so

the old one which was just the chat GPT

uh low IQ person is this and we've added

on the roll prompt and we've also added

in a section for how it should approach

a task a step-by-step Chain of Thought

prompting method that we've Incorporated

next we have the specific section which

is below the task and this is really an

addition to the task so to not get it

too bloated on the task component you

can then have important bullet points

that reiterate uh more instructions or

important notes regarding the execution

of the task so using the example of the

Outreach message generator prompt

examples of specifics what this might be

each message should have an intro body

and outro with a tone that's informal

use placeholders like this so it's kind

of a list of additional points that

outside of just the core part of the

task you can give additional uh kind of

bullet points which is pretty handy uh

when you're modifying The Prompt when

you're editing it if you think it's not

doing something correctly you can just

easily add another bullet point on so

this is kind of what I will do most of

my modification when I'm writing my

prompts and the tech associated with

specifics is called emotion prompt and

this refers to adding short phrases um

containing emotional stimuli emotional

stimula emotional stimula right to

enhance the prom performance so here's

the research results emotional stimula

can be things like this is very

important to my career this task is

vital to my career and I really value

your thoughtful analysis this continues

on from role prompting a bit cuz you're

kind of continuing to hype this thing up

and say look like you I really

appreciate how how good you are at this

thing and and you being part of this

business and what we're doing is so

important and it has massive

implications on myself and my business

and also on society as a whole the more

you can hype it up and tell it that is

its task is like the world is going to

fall apart if it doesn't do this thing

right the better the performance you can

get out of it so the research results

here are adding emotional stimula which

can be as short as these two little

phrases here this is very important to

my career um and this is vital to my

career these little lines here uh

increased 8% on simple task and 115% on

complex task compared to zero short

problem

so huge increase on complex tasks which

is likely what you're going to be

building your problems for anyway and it

also enhanced the truthfulness and

informativeness of llm outputs by an

average of 19 and 12% respectively so

not only are you getting the increase in

accuracy is is this thing getting the

right uh the right output in the right

response but also it's more truthful and

informative which is me fluffy things

but more being more truthful and

informative is probably a good thing

right so the ROI just adding a few of

these words for the performance of your

prompt is ridiculous there's no reason

you shouldn't be throwing in a couple

these emotional kind of lines which is a

this is very important like this is such

a key thing in the business that you are

part of so the key takeaways here adding

simple phrases like these can encourage

the model to engage in more thorough and

deliberate processing which is

especially beneficial for your complex

tasks that require more careful thought

and Analysis so how does this actually

add into our prompt we have it below the

task section here I can zoom in and we

have the specifics this task is critical

to the success of our business if the

email contains blah blah blah blah and

it's just a list of additional

instructions and we can throw in that

emotion prompt in there as well so

that's specifics you can see it's sort

of coming together here then we jump

into context this is kind of

self-explanatory but just giving the

model a better idea of the environment

in which it's operating in and why can

be helpful to increase performance and

this also gives us an opportunity to

really further instill the role

prompting that we did at the start and

also the emersion prompting that we've

done in the specific so an example here

from our email classification system

could be our company provides AI

solutions to businesses across various

Industries but Accord about who the

business is we receive a high volume of

emails from potential clients through

our website contact form Your Role again

role prompting we're incorporating again

reminding it of the role that it has is

classifying this emails is essential

emotion prompt for our sales team to

prioritize the efforts and respond to

inquires inquiries in a timely manner by

accurately identifying motion prompt

again Etc so you can read the rest of

that but we're we're heading up with a

ro prompt again we're giving it context

on the system that it belongs to and

here's here's my general notes I'm

getting here to myself but General notes

for context is to provide context on the

business including the types of

customers types Services products values

Etc then you can provide context on the

system that it is part of as you can see

here we're saying this is part of our

sales process and we get a lot of emails

and then you can provide a little bit of

context on the importance of the task

and the impact on the business um so you

directly contribute to the growth and

success of our company therefore we

greatly value your careful consideration

and attention to classification so just

kind of reiterating a lot of the stuff

that we've done in the role and also in

the uh in the specific section as well

here's the before and after we've added

this context section section down the

bottom uh not rocket science the example

section kind of self-explanatory but we

want to give examples to the model on

how it should perform and and how it

should be replying to it so you given

input output pairs is what you usually

refer to them as um and this goes on to

the technique of few shot prompting uh

single shot one shot prompting um and in

this case we're going to be talking

about few shot prompting because that's

giving more than one example so uh I'll

give you a little bit of a a look into

the research results here um now all of

these research results attached to

Scientific papers that i' I've gone

through and and found and and put in

here for you so if you want to get

access to all of those research papers

I'll put it on a figma or put it on in

the description so you can have a look

at the papers themselves I'm not pulling

these out of my ass uh these are coming

from papers where people have actually

studied these things so um and this

graph here shows the effect of adding

these input output examples on the

performance and accuracy of the prompt

so zero shot prompting is on the far

left we have 10% accuracy for these 175

billion parameters version of gpt3 as

soon as you add one example to this it

jumps up from 10 to nearly 50 to 45%

accuracy and then we get sort of a a

diminishing returns as we continue to

increase up to here is 10 examples so

this is 10 input output pairs so a QA QA

QA one QA and one example of an input

and an output that is a a a shock with a

one shock prompt we got a 45% accuracy

and as we got up to 10 we got a 60% and

kind of flattened off after there so the

research results uh is that GB3 175

billion parameters achieved an average

14.4% improvement over its zero shot

accuracy of 57.4 when using 32 examples

per task so that's way up here um and

using a lot of them and it kind of crept

its way up uh but for us the key

takeaways is that providing just a few

examples literally going from zero

examples to one massively increases the

performance compared to zero shot

prompting when it doesn't have any

examples so accuracy scales with the

number of examples but it shows

diminishing returns most of the gains

can be achieved between uh 10 to 32 well

crafted examples and personally I go for

like 3 to 5 I don't really want to be

sitting there all day writing all these

examples and the more examples you give

the more tokens you're putting in the

input of your prompt and therefore the

more expensive it is every time every

time you call that prompt so if it's

part of this email classification system

and we have 32 examples we're going to

have 32 examples worth of context and

token usage in our Automation and that

means every single time an email comes

in it's going to be sending off huge

amounts of tokens uh as part of the

input and going to be charged on those

import tokens as well so 10 to 32 is is

a sweet spot according to this paper

just do 3 to 5 it does a job enough um

and at least in my experience and and

the stuff that we do at morning side as

well so a little bit more on examples I

won't bore you too much here but this is

kind of the key part here that these

guys doing these these uh these papers

and doing the research they documented

roughly predictable Trends and scaling

and performance without using fine

tuning so by giving examples you are

kind of impr prompt fine-tuning these

models uh and people talk about fine

tuning and everyone thinks that you need

to do it I personally for me and my

development company we build these AI

solutions for businesses and we've never

had to use fine tuning because we're

actually good at prpt engineering and

there's only a very limited number of

use cases where fine shunting actually

gives you an advantage um and that's

just from our experience so if you want

to avoid doing the messy stuff of data

collection and fine tuning and all that

crap uh just get good at prompting get

get good at writing these examples and

you can achieve the roughly similar uh

performance increases um as fine tuning

without fine tuning so this graph here

shows an interesting uh bit of data that

I do want to share is getting a little

bit Ticky but uh this graph on the right

here shows a significant increase in

performance from zero shot which is the

blue to few short completions so if you

add in some examples you're going to

jump up from I think it was 42 up to

nearly 55 60 a big jump immediately just

by adding a few examples but

interestingly the gold labels here so

these orange pillars these orange bars

uh that refers to the tests done where

the labels were correct so maybe if the

email classification was um here's the

email here's classification and we gave

it correct examples the performance

increase within the study was shown

regardless of whether those labels were

correct so this tells us something

interesting that the llm is not strictly

learning new information so by giving us

giving it few short examples that have

the correct labels it's not necessarily

learning that information it's actually

just learning from the format and

structure uh and that helps to increase

the accuracy of the outputs overall the

accuracy of the label itself does not

actually appear to matter too much uh on

the on the overall performance so you

can have incorrect labels and it's still

going to perform just as well um because

you've given it some examples on how it

should respond so long story short

throwing in three to five examples is

going to greatly increase the accuracy

and the performance of your prompt um

and it's also should be thought of more

as teaching it how to structure the

output so this is very important if

you're not getting the structure you

want and throwing in a whole bunch of

other rubbish like oh well this is the

answer to the question if you just give

it a few examples of how it should

respond it's going to look very closely

at that and it's going to perform much

better for you so think of it as fine

tuning of the St the tone and the length

and the structure of the output um and I

think this is something that a lot of

people miss out on when they don't add

these things in because it's it's so

important if you just wanted to give you

one word and you kind of try to tell it

in the task to just give one word

responses sure it might listen to it but

if you give five examples of input and

then just a one word output like in our

case opportunity or or needs attention

or ignore these labels for our email

classification system uh it's going to

perform so much better so here's a

before and after again we're getting a

little bit small here so I'll allow you

to pause this on screen as you wish but

we've given it a couple examples you can

see how I've done it here in this case

it's email label um I usually tend to go

for a q and

a uh that's usually my go-to strategy or

input output um but that's that's

basically how we do it we go example one

uh we give the QA and then we give a

space example two some you don't even

need to put these on um you can just

leave it as that and it sort of figures

it out uh but that's that's F shot

property and examples and how we've

compared them

now getting on to the final bit stick

with me because you are learning some

very good stuff here uh the notes

section is the final part and this is

our last chance to remind the llm of key

aspects of the task and add any final

details or tweaks uh this is something

that you'll end up using a lot as you're

actually doing the prompt engineering

workflow um in the list I usually end up

having things like output formatting

notes like you should put your output in

X format or do not do X like if it's

doing something as I do a test this is

kind of where I'm iterating on the on

the prompt so if I if it gives me an

output and it has doing something way

wrong or just say at the bottom at the

note section say do not do X or you are

not supposed to do this never include it

in your output uh these kind of things

are very easy to slap onto the note

section at the bottom um small tone

tweaks reminders of key points from the

task or specifics is really what I use

the note section for um and and as I say

here it usually starts out quite skinny

because if you do the all the prompt

incorrectly you'll have well I've got

nothing else to say in the prompt all

I've got nothing else to say at this

bottom section then you give it a spin

you throw some inputs at it and it

starts doing some wacky stuff and you

come back and go oh well this just

reminded of some things I've said

earlier on and you start to add this

list of things to the notes now don't

let it become too long u because it's

going to start to sort of water it down

you'll notice that it'll start

forgetting earlier notes if you put too

many notes in um but less is more here

and if it's it's really just to tweak

these outputs to to get the right right

kind of responses without refactoring

the whole thing and restructuring how

you did the task in the specific so it's

just kind of a lazy way of tacking

things on to just get it nudged towards

where you want it to go um now we have

the note section and it's based off the

Lost in the middle effect which is from

another scientific like research paper

um and this lost INE middle effect is is

most famous kind of for this graph here

uh which shows that language models

perform best when relevant information

is at the very beginning Primacy I'm

learning new stuff here as well or end

recency of the imput context so

performance significantly worsens when

the critical information is in the

middle of a long context and this effect

occurs even when the models are designed

for long input sequences so yes gbt 4

32k back in the day was designed for

32,000 tokens but it didn't really

listen to anything in the middle um

luckily the models that we work with now

um are much better at retrieving

information over large context um but

you should still keep this in mind

because it still seems to apply um and

this is why the note section is at the

end this little graph here basically

shows you that uh when you place the

information at the start the accuracy is

higher and when you place it in the

middle the accuracy is lower and when

you place it at the end the accuracy is

higher but not as high as the start so

it really listens to the stuff at the

start so the role prompt it takes it

very seriously and that's why we have

our task up the top as well that's why

we have the context in the middle

because it's not as important so see

he's starting to knit together all this

information understand these how all

these different uh techniques knitten

together so the way that I've structured

this prompt and the way my team have

structured it I'm going to really re

retelling you what we do at morning Side

by adding these things all in together

uh you see how it starts to fit together

into a proper strategy and not just

throwing over the wall and having some

kind of prompt formula it's actually

based off the science um and and if I L

to talk about science these days so uh

that is lost in the middle I think have

a little more

here the research results of course that

you've been anxiously waiting for is

that when a relevant document is at the

beginning or the end of a context GPD

345 turbo achieves around 95 around 75%

accuracy on a QA task um an increase of

20 to 25% compared to when the document

was placed in the middle um so the key

takeaways from this is instructions

given at the start and the end of The

Prompt are listened to by the LM far

more than anything in the middle um for

this reason the note section is a handy

to append reminders uh for anything that

happened in the task or the specifics

that you notice it maybe isn't listening

to and you need to reiterate um but be

aware that increasing the context length

alone does not ensure better performance

still having less context or fluff will

mean the remaining instructions are more

likely to be followed so while lost in

the middle refers to okay where should

we put where should we structure the

prompt to include uh the right

information to be listen what's the most

important thing in the prompt and where

should we put it yes that does that but

it also it also gives us information on

how we should try to keep our prompt as

short as possible because it's over

longer context periods that these things

start to get bad so the shorter you can

keep the prompt in general it could

listen to the whole thing very very well

but as soon as you've like really made

it bloated um it's going to be losing

some of that stuff in the middle so less

is more um and having less less fluff is

always going to make your your prods

perform better so here you can see in

the note section uh please provide the

email classification label and only the

label as your response so again

reiterating the format we want the

output to be in um do not include any

personal information in your response if

you're unsure uh on the side of caution

and assign the needs attenti label so

little reminders as we've gone through

and and we tweaking this email

classification prompt you will add those

things at over time so getting back to

this little diagram here we have the

role prompting covered off you know how

to use that technique is tell it a roll

and and tell it how good it is at that

role Chain of Thought give it a list of

things that it should do and how it

should break down the the task motion

prompt tell it how good it is tell it

how important everything is that it's

doing few shot prompting give it

examples that it knows the kind of

output format you want lost in the

middle kind of tells you how to

structure everything and where to put

the right information and you can add on

a couple little uh things at the bottom

so that it really listens to them at the

end and finally here we have markdown

formatting man I'm talking at a mile

here and I'm getting really hot anyway

markdown formatting is kind of the final

piece of this puzzle and tied all

together and I learned this from a CTO

Spencer he put me onto this technique

and I use it all the time now so uh

markdown formatting is a way that we can

structure our prompts um for both our

sake so that it's more readable CU When

you write these large prompts it can get

a little bit and like there's a lot of

stuff going on so for our sake it allows

us to structure the reprompt better but

also it allows the llm to understand the

structure a little bit better as well

while I don't have any research to back

that up uh my only data on why we should

be doing this and why it may perform

better is because you can see over here

uh someone managed to extract out the

system prompt from th 3 within chat GPT

and open AI themselves are actually

using uh using these the smart

formatting so you can see uh a pound

symbol here and then tool so these are

marked out headings as we're going to go

into in a second but if open AI is using

it um to train their systems and to to

prob their own systems we should

probably be using it as well which is

kind of why we're doing it here so uh

basically markdown gives us a few new

tools to structure um you may notice if

you're writing a prompt you just got PL

text you don't have any any method to to

Signal what a hitting would look like or

what bulb would look like but markdown

gives us uh those those techniques so we

have hittings uh hitting one is the

largest hitting two is the second lest

hting three is the third lest so you

have now different layers of hittings so

you can have like roll task all these in

the hitting one so just H one as a as a

pound symbol and then and then a space

and then whatever you want after it

which you'll see in a sign um but then

if you have little subsets or

subsections like examples hitter and

then you want example one you can have

example one as a hitting three or a

hitting two so you have different layers

of hitting and importance uh you also

have bolds italics underlines list

horizontal rules and more so if you want

to jump into the fancy stuff I'll teach

you the basics here of markdown but you

can also do these other things I'm not

sure what the effectiveness is um of

bolds and italics and stuff but I tend

to just use the use the headings as a as

a structure tool so key takeaways on

markdown formatting is use these H1 tags

single pound symbol uh to Mark each of

the components for your prompt and then

you can use the H2 or three tags or even

bolds and stuff to sort of add add

additional additional structure to other

parts of it so here's example of how you

should add it in hitting one roll

hitting one task specifics context and

then Within context I've added in here

look you might want to break the context

into subsections of okay let's use a

heading 2 and go about the business

about our system so you don't need to do

that all the time but this is how you

can start to use other types of headings

in like H2 or H3 tags to to split up uh

some of the other subsections under each

of your main headings and then again

examples we can have an example one as a

as a heading three and give the examples

and the notes so that's roughly and you

come in here and obviously you are a BL

blah um

generate BL BL blah you get what I'm

doing you get what I'm saying and so

what this all looks like when we tie it

together um we now have our completed

prompt which this is the before remember

this is where we started this is the uh

the the the super guy who doesn't not

had a prompt this is what we started

with and this is what we have after when

we apply all of these techniques now

this is a little bit overol for an email

classification system but what I want to

show you is that this is how you would

apply it to a simple task like this so

we have the roll that's wrapped in the

AG one tag we have H1 tag here

Etc um and we have all of these

different components role task specifics

context examples and notes all

integrating the uh techniques that we've

been over in this video and now stacking

up all of the increases in accuracy that

we get from these different techniques

we can see that we don't know how much

markdown formatting gives us uh but the

total is potentially above 300% increase

in accuracy then the final step here is

we can add up all of the different

increases and and the performance

increases that we get from these

techniques and we can can sum it up to a

300% or more increase in in performance

so me you can listen to me or you can

just ignore it or you can use these

place by Place wherever you think you

need it um but considering emotion

prompting is literally just a few words

saying you're the best and this is

really important to me and Ro prompting

is like one or two lines and lost in the

middle is really just more of a an

understanding of where to put the right

information you prompt you've now got a

toolkit and going back to this guy over

here look at this guy he's got a toolkit

he understands the science understands

from research papers at why these things

work the way they do and because he has

this this deeper understanding of what

makes llms do the right things that they

want them to do he's better able to

perform and as you can see he is on the

upper end of the spectrum here so this

is the guy that you should be now all

you need to do is take these and apply

it and you'll start to see and and

connect them go okay okay so lost in the

middle um that's not doing what I want

maybe I need to change the stuff at the

start and the end okay uh it's giving me

the wrong structure and style okay maybe

maybe I give some more F short examples

of how it should be responding and I I

take my time and I write them carefully

and I tell them the kind of style and

structure of the response I want it's

really not rocket science and people

have already done the hard work by doing

the the research to get these kind of

results so um to wrap up this video I've

given oh actually we have a

considerations page here uh context

length and costs as I mentioned earlier

for high volume tasks um like this

example of email classification system

uh I guess it's not too high volume but

if this thing is doing like 50 50 100

reps a day it's really being put through

the ringer and there's a lot of volume

going through the task that you're

building you need to focus on making

that prompt as short and succinct as

possible uh because every time you run

it you are charged for the input and the

output tokens so while you may only be

outputting a label in this case of just

needs needs work new opportunity or

needs attention or ignore you're also

charged for the input tokens as well so

all the prop that you put in you're

going to be charged for plus the

inserted variables as well so you've got

the prompt then you're inserting the

email context you're getting all of that

information and that over you're you're

going to get charged on that so uh keep

in mind that if you're doing a lot of

volume try to use a a cheaper model as

we're going into next but also keep the

The Prompt shorter as well the choice of

model is important as well better prompt

engineering and the skills that I've

just taught you on this going back to

this guy here he has better prompt

engineering skills and can get better

performance out of Cheaper models this

guy doesn't have the skills so he relies

on the more expensive and slower models

which are not good for the client um to

get the performance that he needs

because he doesn't have the skills to

get it to do what he wants and that

brings me back to this choice at model

point which is where possible you need

to use your skills and use your

advantage to bend the cheapest and

fastest model to execute the task

successfully so 3.5 turbo is basically

free like this thing open AI has made

that so cheap and whatever whenever

you're watching this video might be

different but the cheapest fastest model

should be your goto and if you can't get

it working there then you can go up but

you have the skills now um if it has

high volume and requires fast responses

this is when your skills will shine

because you can create prompts that do

and perform um fast and cheap then we

have the temperature and and other model

settings if you're doing creative rating

adiation Etc then test higher levels so

0.5 to1 uh but anything else if you're

putting systems like this whereas

classification or AI is kind of doing a

a a fixed piece of the of the puzzle uh

you want it to be on zero just have that

we're trying to fight against the

inconsistency and and natural randomness

of these models and in order to do that

we need to uh set that temperature to

zero and that's going to make the system

a lot more consistent uh so zero is what

I typically use for basically anything

apart from creative writing cutter uh

script rting prompts the other and the

other model settings like frequency

penalty and top PE are not needed in my

experience just play around with the the

temperature that's all you need to worry

about what I'm going to jump to now is

actually having a chat with my CTO

Spencer um and he's going to share what

we've done at morning side on one of our

projects where we had to go from GPT 4

uh which was doing the job great and

then the client wanted to change to GPT

3.5 turbo to save money and then we had

to kind of rebuild everything in order

to get it working so uh we're going to

jump to that and you get to here for

Spencer again lot smarter than me and a

lot of the stuff that I'm sharing

actually came from what he's learned uh

learned on the job and what he does at

warning side so everyone if you haven't

met Spencer already this Spencer my CTO

he's a lot smarter than I so I'm

bringing him on to chip into this prompt

engineering video just briefly because

um a lot of the stuff that I've just

told you about has actually come from

has big brain here he's been sharing a

lot of the the research papers

particularly within our slack across the

companies we're on the same page so

Spencer I wanted to bring you on here

particularly because we've been working

with a one of our biggest clients ever

today U and I want to particular focus

on how I was talking in this video about

the pr engineering skills allowing you

to get more out of uh lesser and cheaper

models um and how we've had to switch

from a gbg4 based SAS that we built over

to a hbt 3.5 turbo and and the

difficulties in transitioning that so if

you just want to um give any notes on

the on the presentation prior but also

specifically on uh getting more out of

these these lesson models really which

is what I'm trying to teach people in

this video yeah yeah definitely so um

yeah it's an interesting one I usually

uh like to try and break things down so

um when going through these path the key

is is that obviously want to use the

cheaper models first so 3.5 comes comes

first to mind um in this case

specifically for this client there's a

lot of complex uh kind of information

that they were synthesizing out of it so

we made the decision to start off with

gp4 um to to make sure that we were

getting the responses that we wanted now

once it kind of got closer to uh to

release there we realized that the the

cost that was Associated um with running

these models is going to be ative so we

had to yeah kind of take that transition

now and and gauge down to 3.5 so

whenever I'm doing that specific task

the key one that I'm looking at is yes

prompt engineering one um and then two

is scope reduction um gp4 is really good

at a bunch of different things uh and

and understanding kind of the hidden

context that uh that's in the words that

you're doing uh 3.5 is is much less so

so um you almost want to break it down

into smaller kind of component size

chunks for the task um and then use

those as kind of contributive to to get

the same results as you would with four

um so that was the steps that we're

taking in this particular project

another good tactic to use as well and

and one that I would highly recommend is

using gp4 first and then taking the

input and output pairings as training

data to fine-tune a 3.5 model as well um

because we found that that's that's

really helpful uh for getting your cost

down but keeping up that GPT for L

quality yeah I'm kind of just bashed

fine tuning earlier in this video

because I say it's it's unnecessary in

almost every case um so I mean using few

short examples is essentially a way of

of fine tuning VI prompting so if you

just give a few short examples of gp4

outputs or human rid outputs would that

not do a lot in terms of getting more

towards the outputs that you're looking

for yeah 100% and you're completely

right on that one fine tuning for I

would say a vast amount of use cases

isn't really NE necessary you can get I

would say 90 even 95% of the way with uh

with just good old fashioned prompt

engineering and and F shot prompt in

here um with f shot prompting there's a

interesting paper that came out last

year um and I can't remember the

specific name of it but uh it talks

about the decision boundary so there's

an important uh kind of lesson to learn

on that is that for the fot prompts that

you're giving the important part is to

give ones that are confusing to the

model itself so the ones that you notice

that it's getting wrong consistently if

you actually categorize those and take

those in and take the one to five artist

examples that you get and then use those

as the uh yeah as the examples in there

you'll actually get a lot of better

results coming out of your model too

well that's that's I'm learning

something on this on this call in this

video as well because uh I mean I'd

always start in my fut show examples

have kind of like the most common ones

you might check a a curve B in there as

well but I just kind of put the five

three to five common ones um but knowing

that we should try to figure out when

it's stuffing up and then and put those

on next examples is great so any other

notes you have on on the content just

Tak a look at the presentation but the

markdown formatting aspect um any of the

other any other techniques I know motion

promps than you want for me so anything

that you got there yeah uh markdown is

one that we use extensively um I'm a

huge ner so I I like writing in markdown

anyways just because most of the the

notebooks uh Jupiter notebooks if

there's any other uh data nerds out

there like myself um so it's it's rather

um yeah

consistent familiar for myself is is any

data or or papers that you've seen with

the uh the markdown base because in the

presentation just before I was like look

I I can't find any research papers but

I'm sure just probably G on but uh it's

more like if open AI using it you'd be

pretty stupid not to do it and even just

functionally for us as as writing these

prompts it's so much more useful to at

least have some kind of structure to it

so purely on our side you'd use it

regardless just to make it easier on

your on your end yeah absolutely so I

definitely remember reading I think at

least a couple papers about structured

uh structured inputs in markdown format

and there's other ones as well that you

can use um but even intuitively so when

they're doing the fine-tuning or fine

tuning in terms of

uh uh reinforcement learning with human

feedback rlf um what they're doing is

they're actually providing markdown

based formatting and that's how they're

structuring these prompts that they're

giving to it in order to fing it so

intuitively of course if it's seen it

more it's going to do better when it

sees more of the same that it's been

trained off um the cool part about using

markdown as well is you get to actually

use semantic information so if you're

writing a Word document if you want to

put bold in there if you want to put

something in italics titles subtitles

all these things it makes it into a much

more structured format and that Nuance

comes through on the other side to be

able to uh yeah make better better

prompts to to get better outputs the

other one that uh I would suggest as

well is they like small little things so

uh being very encouraging towards uh an

llm can help so uh I usually start off

with you're a world class X and you know

you are an absolute star doing this it

seems a little bit ridiculous at the

time that I'm not getting this positive

feedback to a machine but uh very

helpful um the other one's telling the

model to take a deep breath and to think

it through step by step before

responding I'm 100% serious has been

proven to actually increase the quality

of your responses and that also doubles

as a as a great one when you're

significant other as is angry usually

that yeah yeah I would not suggest that

as a as a I'll be honest follow the

chcken by calm

down anyway it's good you mention that

sorry that the the hype in the model up

I talked about this just earlier in the

video is that look this a motion prompt

thing where you can get I think 115%

increase in your in your accuracy it's

just by being like wow you well firstly

on the role prompting being like wow you

are like the best at this and then

providing enriching it with additional

words to to reinforce like how good it

is at that toas and then the other I

think so um let M anyway back to what

you said yeah I and it's actually funny

as well Persona based uh thing so if you

uh not only tell it it's a world class X

if you actually use names of specific

people especially people who have

written over the Internet or uh you know

if you say you are Albert Einstein

it will actually come out with higher

quality outputs um that are very much in

the style of writing the the person that

you're talking about I use it for

programming personalities so Theo he he

does the T3 stack um and I'll constantly

say you're Theo show me how to refactor

my code like the wood and and that

actually goes really really well um and

then the other kind of last one in here

is on the positivity rout but not using

negative uh feedback for so a lot of the

time your your first impulse is going to

be like stop doing this don't do this

don't do that if you instead focus on do

this or do that um the negative conent

uh words actually are associated with

worse outcomes than positively France

yeah it's just interesting because then

the in the research for this and I was

trying to put together okay like

negative prompting is this a real thing

it seems like the consensus is that it

doesn't actually uh do much but I will

I've anecdotally

the contrary which is uh if if it's

doing something incorrectly I'll usually

just put at the very bottom in the notes

section just never do this in your

output and it usually tends to work so I

mean there's both sides there it works

for me sometimes but it's probably

something a lack of my skills as well um

that I should be doing it further up but

yeah there's some really good things I

think if you guys can as B said that's

another GM that I'll be I'll be

incorporating into my prompting is

giving it a name giving the rle a name

um and that's something OB you just say

you're an expert this this this um but

if you have an example of a real person

or that someone that the internet would

have had information about um you can

throw that in there as

well yeah absolutely um yeah I think

those are the the big topl line ones for

me at least right yeah no that's really

helpful again this is why I brought

Spencer on even I've I've learned

something here um but yeah we can jump

back to the video thank you Spencer

thanks so much then so I hope that's

drilled in the importance of PR

engineering and and being able to use

these cheaper and faster models to

achieve the outcomes that your clients

want otherwise you're not going to make

any money uh but going back to to this I

just want to say look everything that

I've just taught you here can be applied

to all these different types of systems

and what I want to leave you off with at

the end of this is examples of things so

an AI agent is is like GPT is are a good

example of this um or the building AI

agents on my own platform in my own

software agentive if you want to check

it out we're only on weight list at the

moment so you can check that out in the

description uh but agentive allows you

to build AI agents as does the gbt

Builder on on the chb site but what we

want to do if we modifying this prompt

formula for this use case of AI agents

is to modify to include how to use the

knowledge how to use the tools and your

answer then you can provide examples of

response styes and Toad so you can pause

that take a look see but here most

important things to point out is that

I've added in U so you can see roll task

specifics and then tools so the tools

here if you are adding custom tools into

your uh into your gpts or into your AI

agents you can add a little section uh

using the same kind of format right we

have a heading and say you have two

tools to use one I like to include the

knowledge base if I've added any

knowledge to my AI agent I'll sell tell

it use the knowledge base because it's

actually that's how it's working they

use it as a knowledge based tool they

just don't already tell you that it's a

it's a tool um so you construct it

knowledge base is one of the tools you

have you can use it when you're

answering AI business related questions

and number two is a coine similarity

tool it could be other tool that's

calling relevance or something uh but

tell it how to use each of the tools

that's involved and then examples of

okay here's a question someone ask the

agent here's how you should respond uh

Etc so not not rocket science you guys

can use that uh but that's how I write

my adapt this formula to do AI agent

prompts and it works really well next is

voice agents you need to modify the

prompt formula to include a script

outline if necessary uh so sylow BL AI

air all these things that are popping

off right now uh you can modify the same

prompt template uh to do uh really good

voice agents for you so role task but in

the task here we're giving you an

outline of how it should talk and the

steps involved uh then we have the

specifics then we have context about the

business uh this is an example for a

restaurant um I'm just giving a bit of

context on the restaurant there then we

have examples of how it should respond

to the most common questions as I said

before you can also come in here and add

in a script section and add in like a

rough outline of how the script would go

but I've kind of included that in this

uh in this in in this section here from

a high level at least so voice agents

same sort of thing modify it to to do

the job then we have ai automations

which can be using zapia make air table

air table now has AI which is cool uh

but you can create powerful AI tasks and

businesses they can be relied upon to

handle thousands of operations a month

uh what we just built in the email

classifier is an example of an

automation so I don't really need to go

over this but here's another example at

the end here you can see sometimes I

like to throw this in um is after I've

given examples at the bottom I'll go q

and then I'll put the constraint in or

in this case the variable uh in again

and then I'll leave the a open up put

space and then it's just going to kind

of autofill that and it's a it's another

technique you can use to to get it to

only output uh the exact kind of uh

output style that you want so feel free

to use that as you need AI tools um you

may not know what I mean by tools but

basically we can set up a bunch of

inputs say Okay Niche offer then we can

insert that into a uh into a into

pre-written prompt and then that's going

to be allowed to connect to either gpts

or you can build it um on on a on a

landing page and it can be used to speed

up workflows so there's so many

different ways you can use it um here's

an example again you can pause that this

an example um here you can see I'm

inserting the variables uh we have lots

of input output Pairs and then I'm

screaming at the end here because

because it wasn't do what I wanted so uh

yeah take those I'll I'll leave a link

to this presentation down on uh I think

it'll be on my school community so you

just find this video um there'll be a a

resource for this thing in the YouTube

Tab and you can find this video pull

this up and then and use this as you

wish so I want to bring you back to this

um here's a lollipop um because you get

a lollipop for now completing this

course and you're now a successful and a

a genius level I'm not even sure what

this guy's supposed to his name is

supposed to be but he looks like a

genius to me he looks like a Jedi or

something cool so you now this guy and

you didn't end up being stuck in this uh

this midb territory so here's your

little lop and I'm proud of you for

getting through this because the skills

that I just taught you as I say affect

every different thing you're trying to

sell in this AI space if you don't have

this nailed um you're not going to be

able to build things and you're not

going to create value for your clients

cuz you're going to have to use even if

you're kind of okay but you can't get

the cheaper model to do what you need it

to do then you're not going to be able

to succeed long term and I mean you put

yourself up if if someone was offering

the same AI service and you said Hey

look it's going to cost you this much

month and it's going to take 10 seconds

to respond and some other guy goes okay

it's going to cost you one1 of that and

it's going to take a quarter of the time

um who's going to win there so as as

much PVP there's not much PVP going on

in the space right now because there's

very few people selling selling a

Solutions at agencies so we're still

very early to it but over time if you

don't have these skills you're going to

get wiped out by people who do um and

yeah keep in mind there's so much

potential to be squeezed out of these

prompts and out of the these models if

you just apply this technique so every

300% increase I'm going to be making a

couple more of these Style videos if you

did like this if you like me being a lot

more uh no and just telling you

outs then let me know in the comments

because I much prefer doing these kind

of videos even though I'm now getting

super hot and ready and my cats here but

I've like making this personally it's a

lot more fun than my normal videos where

but uh yeah you get the idea if you've

enjoyed please let me know down below

and uh subscribe to the channel if you

haven't already I'm probably going to

have a couple more videos like this on

core things that I think you need to

understand because if you don't learn

this then you can't use my sass and I

can't make money so I'm very selfishly

teaching you this stuff so that one day

you can use my sass and I can sell my

sass for hundreds of millions of dollars

so forgive me for being selfish but you

get to win along the way um but yeah see

you in the next one