Getting Accurate AI Answers in Voiceflow

Summary

This video script offers a structured approach to enhancing AI assistant responses by refining data structuring. It highlights common issues like outdated information and inaccurate answers, then outlines four steps to improve AI performance: identifying key information types, selecting up-to-date and relevant data sources, ensuring proper data formatting, and adjusting knowledge base settings for optimal AI response accuracy. The guide is aimed at building a robust AI assistant capable of providing consistent and reliable answers to user queries.

Takeaways

• 🛠️ Understanding how to structure your data is key to improving AI responses.
• 💡 The AI can give incorrect or outdated answers if it's pulling from the wrong data sources.
• 🎯 Start by identifying the core topics users will ask about, like pricing, case studies, and support documentation.
• 🔍 Less is more: Focus on including fewer, higher-quality data sources in your knowledge base.
• 📊 Properly format your documents before importing them to ensure the AI can interpret them correctly.
• 🔄 Use integrations like Zendesk to import up-to-date support documentation directly into your AI's knowledge base.
• 📂 Utilize plain text and specific data sources for highly accurate information, such as pricing details.
• 🤖 Inject frequently asked questions (FAQs) manually into the knowledge base to ensure consistent and accurate responses.
• ⚙️ Adjust settings like temperature, max tokens, and chunk limit to fine-tune the AI's response behavior.
• 🔧 Regularly test and refine your AI's responses to ensure the highest quality answers, starting with the most important information.

Q & A

What is the main issue with the AI answers discussed in the video?

The main issue is that the AI answers are sometimes incorrect or not useful, even when the relevant information is available in the knowledge base.

What is the first step suggested in the video to improve AI answers?

The first step is to understand the types of information needed and where to source them from, ensuring that the core questions people ask are covered with up-to-date information.

Why is it important to focus on pricing information when structuring data for an AI assistant?

Pricing information is sensitive and often a key topic of inquiry for users. Ensuring it is accurate and up-to-date is crucial to avoid misinformation and maintain trust.

What is the significance of case studies and use cases in the context of the AI assistant's knowledge base?

Case studies and use cases provide real-world examples of how the AI assistant's platform, in this case Voiceflow, can be used, helping users understand its capabilities and potential applications.

Why is support documentation considered a large bucket of information for the AI assistant?

Support documentation is a large bucket because it contains a wealth of information that users frequently seek to understand how Voiceflow works and to resolve their issues.

What does the video suggest regarding the source of up-to-date case studies for the AI assistant?

The video suggests manually selecting specific, recent, and relevant blog posts that highlight the latest features and successful use cases of Voiceflow.

How does the video recommend handling the integration of support documentation into the AI assistant?

The video recommends using an integration, such as the one with Zendesk, to pull in support documents as text documents, ensuring they are formatted well and up-to-date.

What is the rationale behind starting with a few pieces of information when building an AI assistant for the first time?

Starting with a few pieces of information allows for easier debugging and optimization of responses before scaling up to include more documents, preventing the AI from getting lost in a sea of information.

What is the importance of formatting documents appropriately when adding them to the AI's knowledge base?

Proper formatting ensures that the information is organized in a way that the AI can understand and access effectively, leading to more accurate and relevant responses.

Why is it recommended to inject frequently asked questions (FAQs) into the AI's knowledge base?

Injecting FAQs allows the AI to provide specific, high-quality answers to common and sensitive questions, ensuring that users receive the exact information they are seeking.

What are some of the knowledge base settings that can be tweaked to improve the AI's responses?

Settings such as the AI model, temperature, max tokens, chunk limit, and system prompt can be adjusted to control the randomness, length, and focus of the AI's responses, as well as to maintain a balance between accuracy and token usage.

Outlines

00:00

🛠️ Structuring Data for Better AI Answers

The speaker addresses common issues with AI-generated answers, emphasizing that incorrect or outdated responses often stem from poor data structuring. The solution lies in reorganizing data effectively. The speaker promises to demonstrate how to improve AI responses in four steps. They also encourage viewers to subscribe for more content on building advanced AI agents in Voiceflow.

05:00

🔍 Identifying Core Data Sources

The speaker discusses how to start improving AI responses by identifying key information categories, such as pricing, case studies, and support documentation. They stress the importance of up-to-date data, especially for pricing, which is crucial for accuracy. The speaker also explains the challenges of importing large amounts of content, noting that AI might struggle with outdated or irrelevant information. They suggest starting with a smaller number of core documents to ensure high-quality responses before adding more data.

10:01

📄 Formatting and Importing Data Sources

The speaker dives into the importance of properly formatting data for AI, especially when dealing with sensitive information like pricing. They demonstrate how uploading documents in formats like plain text or PDFs, rather than relying on web scrapers, can yield better results. The speaker also highlights the control gained from using structured data, which allows for accurate AI responses. They stress the need for adding high-quality, specific information to improve the AI’s understanding and response accuracy.

15:03

🤖 Fine-Tuning AI Responses with FAQs and Settings

The speaker explains how to further refine AI responses by injecting frequently asked questions (FAQs) into the knowledge base. This helps the AI provide specific, accurate answers to common queries. They also go over the settings in Voiceflow that affect the AI’s behavior, such as temperature (controlling randomness), max tokens (limiting response length), and chunk limit (number of data chunks used to generate responses). The speaker emphasizes balancing these settings to improve accuracy and consistency in AI-generated answers.

Keywords

💡AI Answers

AI Answers refer to the responses generated by artificial intelligence when interacting with users. In the context of the video, the presenter discusses how these answers can sometimes be inaccurate or outdated, emphasizing the importance of structuring data correctly to improve the quality and accuracy of the responses provided by AI systems.

💡Data Structure

Data Structure involves organizing and formatting information in a way that is easily accessible and understandable for AI systems. The video highlights the importance of data structure in enhancing the performance of AI models, suggesting that well-structured data leads to more accurate and reliable AI answers. Examples from the script include organizing pricing information in a format that the AI can understand and use effectively.

💡Knowledge Base

A Knowledge Base is a collection of information that an AI system uses to generate responses. In the video, the speaker discusses the challenges of maintaining an up-to-date and relevant knowledge base, pointing out the need to selectively include accurate and current documents to improve AI responses. The script illustrates this concept with examples of outdated blog posts and pricing information affecting AI answers.

💡Voice Flow

Voice Flow is a platform mentioned in the video used for building advanced AI agents and voice assistants. The speaker uses Voice Flow as an example to explain how data integration and structuring can impact the effectiveness of AI systems. The video emphasizes the importance of ensuring the voice assistant provides accurate and helpful information to users.

💡Data Sources

Data Sources are origins of information that feed into the AI’s knowledge base. The video discusses different types of data sources, such as websites, blogs, and support documentation, and their impact on the accuracy of AI responses. The presenter emphasizes the need to choose reliable and current data sources to ensure that the AI system provides correct information.

💡Integration

Integration refers to the process of combining various data sources into a single system for the AI to access. The video mentions integration with platforms like Zendesk to ensure that support documentation is current and accurate. This helps maintain the AI's knowledge base with relevant and up-to-date information, improving the quality of responses.

💡Pricing Information

Pricing Information is the data related to the cost of products or services. The video discusses how incorrect or outdated pricing information can lead to inaccurate AI responses, stressing the importance of having a single, controlled document for pricing that is regularly updated to prevent errors in user interactions.

💡Support Documentation

Support Documentation includes guides and manuals that provide users with information on how to use a product or service. In the video, the speaker highlights the importance of having up-to-date support documentation integrated into the AI's knowledge base to answer user queries accurately, showcasing this through the integration of Zendesk support documents.

💡Frequently Asked Questions (FAQ)

Frequently Asked Questions (FAQ) are common questions users ask about a product or service. The video suggests adding FAQs to the knowledge base to provide quick and accurate responses to common queries. By manually injecting FAQs, the AI can give specific and high-quality answers to frequently asked questions, as demonstrated in the video with a refund policy example.

💡Chunk Limit

Chunk Limit refers to the number of information segments the AI considers when generating a response. The video explains how adjusting the chunk limit can balance the accuracy of AI answers with computational efficiency. By setting an appropriate chunk limit, the AI can use relevant data without overwhelming the system or mixing conflicting information, thus improving response accuracy.

Highlights

Understanding how to structure data is key to improving AI responses.

Outdated data sources can lead to incorrect AI answers; ensuring up-to-date information is crucial.

Start by covering core topics like pricing, case studies, and support documentation for better AI performance.

Less data, but with higher quality and relevance, results in more accurate AI responses.

Manual curation of content, especially case studies and blog posts, is necessary for maintaining relevance and accuracy.

AI struggles with information in non-text formats like images; structured text is more effective.

Integrations with tools like Zendesk ensure that the most current support documentation is used.

Uploaded files or plain text provide the most control over the AI's knowledge base.

The importance of properly formatting sensitive data, like pricing, to avoid miscommunication.

Injecting frequently asked questions (FAQs) into the knowledge base can enhance response accuracy.

Modifying knowledge base settings, like chunk limit and temperature, can fine-tune AI responses.

Starting with a smaller set of documents allows for easier debugging and optimization.

Using a single, controlled source for sensitive information like pricing minimizes errors.

Manual overrides in the knowledge base can improve AI accuracy for specific, sensitive questions.

The system prompt setting allows you to define the AI’s persona and response style, further customizing its output.

Transcripts

you're noticing that your AI answers

just aren't good even though you've got

the information in your knowledge base

in fact sometimes they're just flat out

wrong now you don't need magic to fix

this the key actually lies in how you

structure your data so I'm going to show

you how to fix that in four easy steps

but before we do that YouTube says that

78% of the people who watch this video

aren't subscribed so if you want to know

how to build the most advanced AI agents

in voice flow hit subscribe below now

let's dive into what you're likely doing

wrong in a couple steps on how you can

start to fix it so here we've got our

voice assistant and I've gone ahead and

loaded in a ton of data sources from our

website to create our bot but if I go

ahead and ask it some simple questions

like how much does voice L cost it gives

me an answer but this answer is actually

wrong and if I really look into it it's

because it's coming from some outdated

blog post in January like 2 years ago if

I asked another question like what are

the steps in voice flow so you would

expect an answer like the button step

the text step all the steps you use to

design your flow or your assistant and

you can see here that it doesn't

actually answer that either it gives me

some something about user testing and

drafting the design this isn't what I'm

looking for now these answers aren't

good but there are a couple ways that I

can go ahead and improve them and

rethink about how I'm structuring the

data that's going to be informing this

assistant the first one is understanding

what types of information I want to have

and where I'm going to grab those from

now I want this to be a baller website

so users are going to come here they're

going to ask questions I'm going to

eventually capture their emails and try

to get demos and leads but the first

thing I want to do is I want to make

sure that I've got my core bases covered

of what people are going to ask so I

know that people are going to ask about

pricing this is a big one that I want to

make sure I have up toate information on

so let's go pricing the second one is

going to be about case studies and use

cases right I know that some of the

common questions people ask me are what

can vool do how much support can vool

automate what is the type of AI agent

that I can build on voice flow so I want

to add in some upto-date case

studies and maybe some relevant blog

posts about some of the most recent

features that we've released now the

third one is probably going to be the

biggest bucket which is actually going

to going to be our support documentation

a ton of users come to our vo website

and they ask questions about how vo

works now we want to make sure that

they've got a really good source

information to get this from so in that

case I'm going to write support docs now

for each of these buckets I want to make

sure that I've got upto-date information

that's being loaded into my assistant so

for my support documentation I'm

actually going to go ahead and grab

these specifically from zendesk I may

also add our developer documentation in

here this ensures that the information

that people are going to be asking about

is as up to date as our docs are so I'm

going to say import from

zendesk now upto-date case studies is

going to be a bit more manual I want to

understand and look through our blog and

figure out what are the specific types

of blog posts that I want to include in

our assistant to make sure that users

have the most up-to-dated information

one of the biggest hazards with

importing your whole website is that you

have a ton of content out there that is

out of date but the AI has no idea

what's out of date or what isn't and so

that's up to you to decide what goes

into there to kind of form the brain or

the base of their knowledge and the last

one here is pricing now pricing is super

sensitive right you don't want to give

out wrong prices you don't want to give

people R this information so I want to

be really careful and make sure that

I've got up toate information and that I

only have one document around pricing so

that I can go ahead and update it if the

pricing ever changes or that the

informational pricing I know is only

going to come from one unique source

that I have a lot of control my

recommendation here is that when you're

building your assistant for the first

time you start off with a few pieces of

information so in this case I'm going to

just start off with pricing and then I'm

going to move to the case studies and

it's for documentation when you start

off with the smaller number of documents

in your assistant it's easier to debug

and figure out how to get the optimal

responses before you go ahead and add

more documentation and you get lost in a

c of information now we're going to go

ahead and start adding the most

important piece of information first

which is pricing and now this goes into

the second rule which is to make sure

that your documents are formatted

appropriately and I'm going to show you

what that means so let's get rid of all

of this other information in our

knowledge space here so we can start

fresh great now I'm going to start off

with pricing so I want to add in

information from our pricing page this

is going to be the most upto-date source

of voice s information and so you can

see here on our pricing page we've got

our actual prices which is 50 bucks a

month for our Pro Plan 500 for our teams

plan and more for our Enterprise but the

first problem here is that this

information isn't actually organized

well for an AI to understand now for a

human it's easy because I can understand

that this is in a kind of form that

these icons mean something but a aren't

as good as doing that on top of that the

other thing is that the web scraper

inside of voice flow is actually built

for text documents so it's not going to

be able to recognize that on this web

page these UI elements actually indicate

different parts it's just going to

scrape all the text from this page and

jumble it together so if I go ahead and

try to add this page directly to the

knowledge base this isn't going to go

that well so let's add data source

URL and import this

so one way you can check is that when

you actually upload a document to the

knowledge base if you click on not the

URL but this white part here it'll open

up a side panel and you can see how

voice flow has chunk that information so

right off the bat you can see that I've

got a bunch of URLs here this doesn't

really make much sense I'm not seeing

any of the columns and that's because a

lot of the information on this page was

actually in images so they didn't get

scraped by the web scrier so by now try

to go and ask something like how much

does voice will cost it gives me an

answer but it's not actually what I want

right it says editors have the ability

to make changes voice has different

pricing plans check the website that's

not why I'm asking your assistant I want

the actual answer and if I check the

sources here it's grabbing it from the

pricing page but because that document

doesn't actually have any well properly

formatted information in it that the AI

can access it's not able to actually

answer it so because this is really

sensitive and I want to make sure this

is right this is where I can add a

different data source with more

information that's structured properly

so let's go ahead and get rid of this

and instead let's add a data source

that's bit different so in terms of the

order of control that you have so URLs

and site map have the leas control this

is using a web scraper to actually

scrape the page for text and then it

tries to put it into the best way it can

for formatting and uploads that to voice

flow an integration is pretty solid if

you're using zenes this is a great one

this is actually going to pull the

documents from zenes and because these

are text documents it's going to be

formatted well the next one are uploaded

file and pl Tex now this is how you're

going to get the most control out of

your assistant because when you upload a

PDF you know that the documents are

going to be structured properly but

really plain text is what I'm going to

go for here because this is going to

allow me to inject q&as or specific

pieces of information into my knowledge

base that I can then go and modify later

so what I'm going to do here now is I'm

actually going to paste in the

information I have about pricing now if

you have a document that's we better you

know I have just an example here but

ideally you have a document with a ton

of detail about these different PL plans

the prices any other information you

want users to know that can be fed to

the AI and so I'm just going to put in a

quick sample here but voice pricing is

as follows Pro Plan teams plan

Enterprise plan who's it for ETC so now

if I go ahead and import this you can

see that when this is done and I click

it that the information in the chunk is

actually great uh this is exactly what I

put in here I have a lot of control over

this and this is the thing with adding

text is that you can actually go ahead

and modify this afterwards but this is

pretty good and now I go ahead and start

asking it questions like how much is

voice flow it's going to be able to

answer with a much higher degree of

accuracy because it's pulling from that

main source so you can see here was able

to summarize all the different plans and

if I ask you something like I'm a new

agency with one person what plan should

I get it's going to be able to interpret

that and recommend a plan for the user

so $50 a month so you can see here even

though I don't have a lot of information

I just got one piece of information but

it's good high quality information my AI

model is able to understand that

interpret it and then respond that to

user questions in an efficient and

accurate manner so in the case of data

sources with AI less is more and more is

actually not that great you want to have

less higher quality sources to be able

to produce higher quality answers that

are just as robust as what you would

expect with the large dollar space so

you want to make sure you spend a lot of

time making sure that your most

sensitive information is formatted

correctly so now that we've got our

pricing data source in there let's work

through those other sources that we've

got over here in our diagram so we've

got our pricing now now let's added some

update case studies and then some

support documentation we're going to

import it from zendesk so if I head over

to the voice website I'm going to now go

to our blog and I'm going to manually

pick out the documents that I want to

upload because again it's going to be

important that I have a high level of

control so for here I'm going to add a

case study with Trilogy that automated

60% of their customer support with voice

flow this has got a ton of really good

information in it about how they use

voice flow how many their tickets they

actually automated and how they were

actually able to use AI to increase the

number of tickets they automated from 6

to 60% in just a few weeks so let's go

ahead and add that in and for this one

because we know what URL is going to

come from we're going to add in the

URL now let's grab two more let's test

this out now and ask something

like what can voice flow help me do and

you can see here that it says voice will

helps you automate customer support

provide accurate answer to customer

questions Etc can you give me some

examples of customers and how they used

voice great and it was able to actually

really pull from up to date so it says

Rome must start up in the flexible car

industry automated theirselves Trilogy

they saw 35% resolution from the first

week so it's giving us a lot of really

good in-depth information here now I

want to get the third piece which is all

of our support documentation so this one

I'm actually going to import from zenes

so we've got all our support

documentation in zendesk and this is

live up to date and there is a ton of it

that can really help our users so now

that we're know we're getting a base of

really good information we're going to

go ahead and add our support

documentation in here to really make

this a robust assistant so what I want

to do is rather than scraping the

website I'm actually going to use the

integration now Integrations are always

best right now we support zenes we're

going to be adding more in the future

but you can actually use knowledge based

API to upload documents as well so if

you have a Content system somewhere

that's going to be much more accurate to

be able to import it and I'll show you

why so I've authenticated and let's add

all our docs so we're going to add the

voal docs we're going to add English we

don't have any user segments that's fine

and we'll add our guides and our

documentation so this is going to add

136 data sources here so now you can see

that it's starting to add a ton of

information in here but because I adding

these in pieces as I'm testing it out I

have a really good sense that I know my

Base information is good and now it's

about to sendes ones so if I go ahead

and click on something you can see that

the chunks are actually pretty good so

if I go ahead and now ask it something

like what are all the steps in voice

flow this gives me a much better answer

than what I originally got this is a

great um answer and I'm pretty satisfied

now knowing that my voice assistant can

ask answer any question about voice flow

it can also answer case studies and

business information as well as a really

sensitive information about pricing now

we're going to get into Point number

three which is what you can start doing

after you've filled out the base of your

assistant which is to start injecting

frequently asked questions into your

knowledge base sometimes there are

questions you get that you want to give

a specific answer to and you want this

to be high quality similar to pricing

but it may be related to other services

refunds things like that that you know

are pretty sensitive pieces of

information that you want to have the

right answers to so let's say I've got a

question like does voice process refunds

so let's see what it says right now and

it says yes voo does process refunds

customer support a refund by emailing

support ato.com and if I check here it

looks like this is mentioned in some of

my help articles but let's say I wanted

to provide more information to this so

I'm going to inject an FAQ into the

database and the way it works is when

you actually ask a question to the voice

of knowledge base it's looking through

all of these different chunks and it's

returning what it thinks is the most

relevant so you saw in that last example

when I asked the question about refunds

it returned three chunks one of them had

a slight mention of refunds the other

ones didn't but I thought it might be

relevant so what I'm going to do is I'm

going to add a data source and I'm just

going to add plain text now we're going

to have an entire built out solution for

FAQs uh coming out in the near future so

if you're watching this a couple months

from releasee and we've got a whole new

section here about FAQ you canor this

but if you're not this is also a great

way to kind of add in or inject manual

overrides into your knowledge space so

I'm going to ask put a question here so

I'm going to say question or maybe Q is

does voice flow

offer

refunds does voice flow give out

refunds and now I'm just going to add in

my own answer voice FL only processes

refunds in specific

cases please reach out to support f.com

if you require a refund great so it's

import this in here and so now I've got

something a bit more specific so I now

ask again does voice flow offer refunds

you'll notice that now it says voice

only processes refunds in specific cases

please reach out to voice.com if you

require refund and if I look at the

chunks that's because the first one it's

Serv for saying that's the most

important is the Q&A that I added in

there because I added a piece of

information to the knowledge piece

that's highly relevant to this topic

area it's going to service this first

and it's most likely just going to use

the information from here and it'll look

at the other ones just to see if there's

anything that it might be related to so

you can see it gave my thing about

pricing here and it has this document

here about downgrading and can

canellation so this is how you can

inject an FAQ into your knowledge base

to have it have a higher degree of

answering accurately for that question

now we're getting into tip number four

which is really about modifying your

knowledge based settings so you've

gotten hints at how the knowledge base

works here about retrieving and

providing chunks and you find more

information into the details of how

Vector databases work in general in the

links Below in our YouTube channel but

let's go into some set settings that but

let's go into some settings that you can

tweak to try and get better or more

accurate

responses within your voice on knowledge

base you have these settings here these

are the global settings that are kind of

like the foundation of your assistant

now if you open these up there's a

couple different things so there's the

AI model so we've got a bunch here we're

adding way more in the future we've got

the temperature so this is going to

control the randomness of the answer

that the large language model provides

if you're okay with your answers being a

little bit more flexible you can

increase the temperature here if you

want them to be pretty standard you know

someone ask the same question three

times you want the same answer and the

same way you can reduce this down to be

very deterministic I'm okay I want to be

a bit deterministic so I'm going to have

it be

0.1 Max tokens are also important this

is the max output tokens so this is the

number of tokens that can be used to

generate a single response so if I have

this at 128 that's going to kind of snit

my answers at 128 but it'll also be a

bit shorter just to be safe I'm going to

increase this to maybe 200 just to make

sure that the AI has enough room to be

able to communicate an answer now the

last one here that's very important is

the chunk limit so if you remember when

we asked a question there are certain

sources that are passed back to the

knowledge base these chunks on the side

here this is going to dictate how many

chunks it should pass to the large

language model to determine an answer

from the more chunks you have the more

information the large langle has to work

with but the more chance that it might

actually mix up conflicting pieces of

information and give the user a wrong

answer so a large number of chunks works

really well when you have an knowledge

base that you know is really consistent

and high quality you're going to pass

information over and the large language

model is going to look at all eight of

these different chunks of information

and assemble an answer now the more

chunks you use the more tokens it's

going to cost you can imagine that 10

chunks is like 10 of these gigantic

pieces here being sent to the AI it's

going to use a lot of tokens may not

always be best on the other hand if you

use one chunk so you lower it

down the AI model will only have one

piece one chunk to to go from and so if

you have relative information in a

couple pieces it's not going to see that

you'll just see one but on the good side

here it's only seeing the most relevant

chunk so this is a good one if you want

to make sure that your answers are very

highly accurate you may have more cases

where an answer isn't found because the

answer wasn't necessarily in the first

chunk that was past but for the most

part you're going to have much more

accuracy on your answers and so that's

what this chunk limit is the default

here is three I like to keep it at three

or two it's a good balance between

accuracy and the number of tokens the

last thing here system prompt this is

pass the AI model as more of like a

general set of instructions so this is

great to enter a Persona and even things

like keep your answers brief or answer

the questions as detailed as possible

and provide a lot of information so

those are the four steps let's go ahead

and wrap this up and actually try out

some of those original queries on our

new and improved voice know face spot

how much is voice flow and you're going

to see that we've got answers that are

much more accurate and up to date what

are all the steps great and done now you

have a much better idea of how to use a

knowledge based in voice loow and how to

get better AI Answers by better

structuring your data that's it my name

is Daniel I'm the head of growth that

voice flow and if you have any other

questions feel free to hop into our

Discord leave question in the comments

or hit us up on any of our socials if

not we'll see you in Discord and on

voice flow and happy building