Azure Search OpenAI Demo - DIY Microsoft AI chatbot with bring-your-own-data | Unscripted Coding

Summary

This video tutorial showcases how to build and deploy a chatbot for business using Microsoft Azure Open AI, emphasizing data privacy and security. It highlights the chatbot's ability to interact with corporate databases and documents, providing instant answers and citations. The deployment process is simplified with Azure Developer CLI, and the video offers insights into customization, scalability, and the associated costs, making it an accessible guide for businesses looking to integrate AI into their customer service or internal operations.

Takeaways

• 🤖 The video demonstrates building and deploying a chatbot for business with minimal programming skills.
• 🛡️ Microsoft Azure Open AI is used for the backend to ensure data privacy and security, unlike Open AI which may use data for further training.
• 📄 The chatbot can interact with and answer questions about documents, such as PDFs, uploaded to it.
• 🔍 It includes features to search within documents and provides citations for its answers, enhancing trust and transparency.
• 📚 The chatbot can be scaled to handle a large number of documents, making it a powerful tool for businesses.
• 💼 It's suggested for internal deployment to manage sensitive business information such as business plans and customer lists.
• 🌐 The deployment process is simplified compared to previous solutions and involves installing Azure Developer CLI.
• 📝 The script guides through downloading a demo from GitHub, setting execution policies, and running deployment commands.
• 💳 The use of Azure services incurs costs, with the video providing a breakdown of potential expenses for the chatbot deployment.
• ⏱️ The deployment and setup process is time-consuming but mostly automated, requiring about half an hour to complete.
• 🖥️ Customization options are available for the chatbot, including branding and design adjustments to fit business needs.
• 🔑 The chatbot offers two modes of interaction: 'Chat' which turns questions into search queries, and 'Ask' which searches the question as-is, catering to different user needs.

Q & A

What is the main topic of the video?

The main topic of the video is demonstrating how to build and deploy a chat bot for business using Microsoft Azure Open AI without extensive programming skills.

Why is Microsoft Azure Open AI preferred over Open AI in the context of the video?

Microsoft Azure Open AI is preferred because it does not use your data for further training of their models, thus ensuring better security, privacy, and confidentiality for businesses.

What feature does the chat bot version shown in the video have that the previous one did not?

The new version of the chat bot has the ability to ask questions about the corporate database and cite sources from documents, providing a more interactive and informative experience.

What is the significance of the chat bot being able to cite sources from documents?

The ability to cite sources adds credibility to the chat bot's responses, allows users to verify the information, and makes it easier to find the exact text within the document.

How many documents can the chat bot handle according to the video?

The chat bot can scale up to handle 100,000 documents, depending on how much data you want to upload to the cloud.

What is the recommended first step for deploying the chat bot?

The first step for deploying the chat bot is downloading and installing the Azure Developer CLI, which is specifically designed for developers.

What is the purpose of running the command 'set executionpolicy bypass' in the video?

The command 'set executionpolicy bypass' is used to allow the execution of scripts that have not been signed, which is necessary for running the development files for the chat bot.

What is the approximate monthly cost for using Azure Cognitive Search in the chat bot?

The standard tier of Azure Cognitive Search, which is necessary for the chat bot, costs about $100 a month.

How long did it take for the deployment process shown in the video?

The deployment process took approximately half an hour, including the time for the search service to be created and for the resources to be properly indexed.

What additional features can be implemented for the chat bot post-deployment?

Post-deployment, features such as custom domain acquisition, password protection through authentication, and document access control based on user roles can be implemented.

How does the 'Ask' tab differ from the 'Chat' tab in the chat bot interface?

The 'Ask' tab searches the question as is and combines the search result with the user's question, while the 'Chat' tab turns the question into a search query, which may lead to quicker access to the right document.

Outlines

00:00

🤖 Building a Secure Chatbot with Azure AI

The video script introduces a tutorial on constructing a chatbot for business use without extensive programming skills. It emphasizes the use of Microsoft Azure Open AI to avoid privacy concerns associated with open AI models that may utilize user data for training. The tutorial showcases a chatbot that can interact with documents, answer questions, and cite sources, providing a secure and efficient tool for businesses to manage sensitive information internally.

05:01

🛠️ Deployment Process for Azure Developer Chatbot

This paragraph outlines the initial steps for deploying a chatbot using Azure Developer Sealy, which includes downloading and installing the Azure developer tools, running commands in a command prompt, and navigating GitHub to download the necessary files. The process requires running PowerShell as an administrator to bypass execution policy restrictions, which is a temporary measure that reverts to normal upon closing the window. The tutorial also provides a command to facilitate the deployment process.

10:01

💳 Understanding the Costs of Azure Services for Chatbot Deployment

The script discusses the financial implications of deploying a chatbot on Microsoft Azure, highlighting the pay-as-you-go model and the need for a credit card during signup. It provides an estimate of the monthly costs associated with various Azure services such as the basic tier apps, cognitive search, and form recognizer. The paragraph also mentions the importance of choosing the right region to avoid service unavailability and provides a link to Azure's pricing calculator for a detailed breakdown.

15:04

⏱️ Time and Process for Deploying Azure Chatbot Services

The paragraph details the time-consuming process of deploying Azure chatbot services, which includes creating various services and indexing data for the chatbot. It mentions the automatic generation of files and the need to wait for services to be fully deployed and reflected on the website. The speaker also points out additional features like custom domains and password protection for added security.

20:05

🎨 Customizing and Branding the Chatbot Interface

The script suggests ways to customize the chatbot's interface, including changing the design to reflect the business's branding with a custom domain, logo, and color scheme. It also touches on the possibility of adding password protection for secure access and the use of Azure Active Directory for authentication. The paragraph highlights the ease of deploying changes with a simple command and the potential for further customization through APIs.

25:05

🔍 Exploring Different Approaches to Document Interaction in Chatbots

The final paragraph discusses the two distinct methods of interacting with documents within a chatbot: the 'chat' tab, which transforms questions into search queries, and the 'ask' tab, which searches the question as is. The paragraph differentiates the use cases for each approach, suggesting that the chat tab is better for general users while the ask tab is more suitable for those who know the specific details of what they are looking for within documents.

Keywords

💡Chat Bot

A chat bot is an AI-powered program designed to simulate conversation with users. In the video's context, it's a tool for businesses to interact with customers or employees without the need for human intervention. The script discusses building and deploying a chat bot using Microsoft Azure Open AI, emphasizing its integration with corporate databases and its ability to answer questions based on the content of documents.

💡Microsoft Azure Open AI

Microsoft Azure Open AI is a cloud-based service that provides AI capabilities to developers and businesses. The video script highlights its use in powering the chat bot, ensuring data privacy and security, which is a key concern when compared to using open AI services that may utilize data for their own model training.

💡Data Privacy

Data privacy refers to the protection of information from unauthorized access or disclosure. The script mentions that using Microsoft Azure Open AI for the chat bot ensures data privacy, as opposed to open AI services that may not guarantee the same level of security for business data.

💡Cognitive Search

Cognitive Search is an AI-driven search capability within Microsoft Azure that allows for semantic search over content such as text in documents. The script describes how Cognitive Search is used to enable the chat bot to search and retrieve information from a corporate database, providing answers to user queries.

💡Form Recognizer

Form Recognizer is an Azure service that can extract text, key-value pairs, and table data from documents. In the script, it's mentioned as a part of the deployment process, suggesting that it may be used to preprocess documents before they are indexed by Cognitive Search for the chat bot's use.

💡Deployment

In the context of the video, deployment refers to the process of setting up and launching the chat bot on a server or cloud environment. The script provides a step-by-step guide on deploying the chat bot using Azure Developer CLI, emphasizing the ease of deployment despite the process's duration.

💡Azure Developer CLI

Azure Developer CLI is a command-line tool designed for developers to interact with Azure services. The script describes using this tool to automate the deployment of the chat bot, including setting up various Azure resources required for its operation.

💡Pricing Tier

Pricing Tier in Azure refers to the different levels of service and their associated costs. The script discusses the cost implications of using various Azure services for the chat bot, such as Cognitive Search, and how the choice of pricing tier affects the overall expense.

💡Document Indexing

Document indexing is the process of organizing and storing documents in a way that makes them easily searchable. The script explains that the chat bot's ability to answer questions is dependent on the indexing of documents, which is facilitated by services like Cognitive Search and Form Recognizer.

💡Custom Domain

A custom domain is a web address that is unique to a business or individual, as opposed to a default domain provided by a service. The script suggests that once the chat bot is deployed, a custom domain can be used to brand the bot's interface, making it more recognizable and professional.

💡Authentication

Authentication in the context of the video refers to the process of verifying the identity of a user or device. The script mentions the possibility of adding authentication to the chat bot to restrict access, ensuring that only authorized individuals can use it.

Highlights

Introduction to building and deploying a chatbot for business without extensive programming skills.

Use of Microsoft Azure Open AI for chatbot operations to ensure data privacy and security.

Comparison between Microsoft Azure Open AI and Open AI regarding data security and privacy.

Demonstration of the chatbot's ability to answer questions about a PDF document.

Inclusion of Bing chat elements and the chatbot's polished demo capabilities.

New feature allowing the chatbot to ask questions about corporate databases.

Explanation of the chatbot's citation feature for tracing the source of information.

Capability to upload and scale document storage for the chatbot's responses.

Potential business applications of the chatbot for product documentation and customer support.

Deployment process overview starting with the Azure developer Sealy installation.

Instructions for running the chatbot deployment script using Windows PowerShell.

Details on setting execution policy to allow script execution for deployment.

Cost implications of using Azure services for the chatbot and associated services.

Breakdown of Azure service costs, including cognitive search and form recognizer.

Estimation of monthly costs for running the chatbot with Azure services.

Description of the chatbot deployment process and the time it takes to complete.

Final notes on adding additional documents to the chatbot's database for more comprehensive responses.

Differentiation between the 'Chat' and 'Ask' tabs in the chatbot interface for varied user inquiries.

Conclusion emphasizing the chatbot's potential and the ease of deployment for businesses.

Transcripts

Welcome, everyone, to unscripted coding.

Today we are playing again with another A.I.

chat bot and I'm going to show you how you can build

and deploy your own chat bot for your business.

With little to no programing skills whatsoever.

Now let me recap really quickly.

Last week we put together this green looking chat bot, and

what's really important is that it runs Microsoft Azure Open AI in the background.

It runs it in the back, meaning that

you're not relying on open AI or chat.

This is really important for businesses because open

AI tends to take your data and continue training their models with it.

Your security and your privacy is is not guaranteed.

On the other hand, Microsoft does.

It gives you the full GPT three or four experience

without the security or confidence level

confidentiality issues that you may run into with open A.I..

Now, it's not perfect.

You definitely won't take a look at the legal document of privacy by venture.

Guess to say that most businesses will be perfectly fine with this chat bot

deployed internally and putting sensitive information,

your business plans, your customer list, all of these things into a chat bot.

I do recommend watching that video.

First, last week's video and it is in the description

below if you want to click in to it

because we talk a lot about these subtle differences.

Now in that particular version you'll see that we can upload a specific file,

so I can upload a PDF and start talking and asking questions about the PDF.

There are elements of Bing chat in here as well, and generally speaking,

this is a more polished demo than the one we're looking at today.

But the version we're looking at today has one

killer killer killer feature,

and that's asking questions about your corporate database.

So I'm going to click this random prompt.

This is just the canned question to demonstrate it,

but it's asking what's included in my plan that's not in the standard plan.

And the AI is coming up with a bunch of different answers here.

Copay, access to telemedicine, medically necessary services.

Blah, blah, blah, blah, blah.

And what's really cool is that it's citing where it comes from

and allows you to click into it and take a look at the text immediately.

And you can see that different pieces of its answers

comes from different sections of the document.

Very, very, very useful.

And I can follow up with questions. So

the suggestion is, does

my plan cover eye exams?

Question mark And it will continue

asking questions and looking at the document.

You might be wondering where these documents come from.

Well, it's included as a sample, but basically it's coming from my desktop.

Are six files here, but you can scale this up to 100,000.

It's really just a matter of how much you want to upload

onto the cloud to to generate these answers.

So in this case, again, it looked at the document,

different pages than what was cited before.

And linking to it.

So that is a killer feature.

If you're thinking about how you can use this in your business,

you can keep all of your product documentation and

descriptions, marketing slips, pamphlets, fliers

all in one folder and be able to ask it questions

very quickly whether as a support agent or as a customer.

You know, how do I fix this error or how do I do this?

Does it do X and Y? Does it have these features?

You can use it for policies like this one.

If you have a lot of these internal policies, you can have a chat bot

that answers questions about anything really

and many, many more that you can do.

So I think this is one really, really killer feature

that will sell the chat bots.

And I'm hoping that one day these things kind of merge into one solution.

But for now you kind of have to pick and choose your pieces

or you have to know kind of what you're doing.

This one well worth it.

I wouldn't mind deploying multiple chat bots as well as a solution.

Now you'll be

happy to know that even though this one looks a little less polished,

the deployment is much

easier than our previous solution.

So I'm going to just jump

right into deployment.

The first step to deploy is you're going to download

and install the Azure developer Sealy.

Now, if you've installed Azure Sealy before, that is not the same thing.

This is Azure developer Sealy specifically for developers

and you'll see that, you know, any desktop will work Mac Linux Windows.

I am on Windows, so you're going to need

to open up a command prompt.

So go into your the search button in your start

bar way down here and run.

Find command prompt or see AMD.

And you're going to get a black and white kind of interface.

Now, I've already done this.

Basically, you copy this installer,

this command, you paste it.

Copy.

Copy. Paste it.

Click. Enter. I already have it.

But you might have to click y afterwards to agree to terms.

This should take no time.

I probably shouldn't have clicked entered because I've already installed at once

but should be buttery, smooth, very easy to install.

Now we'll come back to the GitHub page, which is a link to my description,

but you can search GitHub, Azure search, open air demo

exactly like here and land on this page.

We're going to click code

and download the zip file.

Now, I've already downloaded it, downloaded it,

and there's a folder in here and I dragged it into my desktop right here.

So this Azure search open, I demo dot main.

That's this folder right here.

So I dragged it into my desktop.

What I'm then going to do is open up

windows PowerShell.

I am going to run it

as an administrator and this is really important.

You have to run it as an admin trader

because an error that I kept running into

is that this is still a development kind of file.

It hasn't been signed.

So what you're going to need to do is allow it to execute.

And I have this command right here

and I should zoom in so you can take a quick look.

It is set execution, policy scope,

process, execution, policy, bypass.

I'm also going to leave a link to this

lengthy explanation about execution policies in the description.

But basically what I'm saying is, just for this one process,

I'm going to allow you to bypass those execution policy restrictions.

This means that

as soon as you close a window, everything goes back to normal.

This may be a little uncomfortable for certain people,

but the part that makes me feel better about all of this.

Let's go. Yes.

Is that again?

This is provided by Azure Microsoft.

Yeah.

They they designed the system.

Now, what I'm going to do

is CD basically change

directories from Windows SYSTEM32

to my folder that I unzipped.

That's that Azure search open.

I demo Dash main and you're

going to see that deployment from scratch.

It's right here.

All you have to do is type AISD up.

Remember A-Z refers to Azure Developer,

so you need to have installed this and that's basically all you have to do.

Get into the right folder, say A-Z app

and you're going to have a couple couple options here.

Now first of all, you need a name.

I'm going to call it

Azure Search,

YouTube, demo, chat bot.

Terrible name, but Will well stick with it.

It's going to initialize

and ask a couple questions about the region and your subscription.

So first of all, you might need to log into Azure. So

I'm making assumptions here, but you do need to go

and get a microsoft Azure account.

You can go and Google Microsoft Azure sign up right here.

It is a pay as you go account.

So this thing does cost money.

You're going to need to enter your credit card when you sign up.

Now, when you do it, you're going to sign up for a single subscription.

You're going to get a bunch of free credits as well.

But I've been using Azure for a little while.

I'm going to choose number three.

This is where I'm spinning a few of my different projects right now,

and you're going

to pick locations just like our last video.

I had issues with Canada because one of the services

called Form Recognizer that takes images

and translates it to tax isn't available in Canada.

So I'm going to switch over to U.S. East

and Alaska stuff.

Let's start money.

How much is this going to cost?

Because as soon as I said, pay as you go, I

that may have raised some concerns.

They do talk about cost action in this read V.

So if you scroll down from GitHub and look at comission,

you can look at Azure's pricing calculator

and click into each of these to understand.

Now they have used a basic tier apps.

This is about 20 bucks a month.

Depends on your region, depends on

other factors as well.

And who knows how pricing will go in the future.

But roughly speaking, this is $20 a month

open air depends on how much you use.

I say for about 20 people, that's using the chat bot on a regular basis.

About 5 to $10

form recognizer gets priced per page,

but there is a free tier as well.

I'm going to leave that alone.

But this could this could cost

ten $20,

especially if you start adding a lot of documents that need to be

that need to have that

form recognized.

Now, here I'm just going to go down into East.

You asked again.

This one is specific for open.

I and I'll make sure they remember my choice.

East U.S.

and let's see here Azure cognitive search.

This is the one that costs quite a bit of money.

We are using standard tier, which costs

about $100 a month.

Quite significant,

but it is absolutely necessary if you're doing this type of a chat bot.

Now, on the previous one where we didn't have a store of documents in the back,

we can get away with using the free tier, but unfortunately

it costs a lot to search your document over and over with A.I.

and this is where I think you're going to get gouged,

because if you are using the standard tier up to 25 gigs,

very reasonable for you to have,

you know,

25 gigs worth of documents.

Soon as you hit the next here, it's roughly four times the price.

So you can say you can think 400 bucks.

It's a lot of money.

So this is the part that's going to cost you, but

it's also driving the most value out of this particular sample.

Next, blob storage monitor.

These are fairly cheap.

I'd say 5 to 10 bucks a month between the two.

I'm probably overestimating it quite a bit.

So you're looking at 130, $140 a month,

all of like the vast majority coming from that cognitive search,

which you're just not going to get away with no matter

which solution you go to.

Okay.

Let's come back here.

If you scroll up,

if it lets me scroll up, you're going to see that

it did a lot of this packaging service.

It's gathering all the data now.

It's creating all the resources.

I like to look at this link

just to see what's happening,

and you'll see that

search service, this one takes a little bit.

So I'm going to pause here.

I'd say it's going to take about 10 minutes to fully deploy

this section, really just waiting on this search service to be created.

So I move this back here

and I'll pause the video.

Okay, So in total,

I waited about half an hour for everything to be done.

Now, to be fair, it took a while, but it was completely hands off.

I didn't have to touch anything.

And if we scroll back up all the way up,

you'll see what was happening.

Now, when we last left off, we were over here where it was

still creating the various services.

Once it was done that it installed a whole bunch of files

and then it started trying

to get all of your data properly,

properly indexed for your chat bot.

So it took each page, sent it to the forum, recognizer or

ran it to cognitive search

and eventually created a vector which your A.I.

chat bot can can make use of. So.

So that took a bad time.

And then from there, it just took some time to actually

gather all of that stuff up and move it into the resources

that we had created at the start again, start to finish.

This took about half an hour very roughly.

And it does take a bit of time, even after it says you are all done

and you have an endpoint here, you will need to wait

another five or 6 minutes for it to actually reflect on the website.

So I'm going to go here, enter the website and.

This looks very familiar.

I'm just noticing now there's a developer setting so

interesting

and we can test it.

So what does a product manager do?

Let's go and try and find an answer for us.

Now, before I start, I did want to go back.

If you remember, this was our list of all the stuff that happened.

I wanted to take a look

at, I think our app service.

This is probably not the right thing.

I'm trying to get go into the resource group.

When you click resource group, you can see all the things that created.

This is your open A.I. instance.

This is your app service.

That's the website document intelligence.

That's your form recognizer search service.

This is where they keep it.

Keep all of your pages and documents and storage account.

That's where you store your raw PDF files.

I wanted to check on the app service

and wanted to point out that this is a very ugly.

You are all automatically generated.

You can go into custom domains

and you can buy a domain right off the bat and call it

cheap for my business

dot com and you can have it a custom domain as well.

The other thing that I think we're not going to do today

is you can put a

a password protection on here.

So if you enable authentication here

you can add ampersand dictation

which you do within the app service.

So over here you can use authentication, you can create

with with Azure Active Directory or apparently it's called

Microsoft Entrance now and put basically a requirement

for you to log in through Microsoft before you can see this Web page.

There's there's a lot of little things, but

one that we're also not going to explore but is very much worth looking at

is this guide that allows you to

separate the documents.

So you can actually have different

different pools of documents.

So, for example, I as a junior,

can only see the basic level document.

Some of our C internal policies for the business and someone more senior.

Again, requiring that authentication is able to use the guide

and see all of the companies leases and

and you know, recruitment contracts

and all the employees

employment agreements, all of those much more sensitive pieces.

You can lock them between and behind different groups.

This is quite a bit of work to get working

from what I can tell.

And and so I'm not going to go into that for a demo, but this is pretty cool.

We asked the next bit, What is a product manager?

And looking at the pages,

I can see manager of product Management.

That is a very odd title.

And all of this with basically one command

and a lot of time in between to get set up.

I have done a video on Bring your own data before

and all of this can be done through APIs and you can create your own chat bot.

So I you don't necessarily need to do it this way,

but this is a one really quick way for your business to get all set up.

And if you were to change

the design, you can go into the app and assets.

So let's look at the app

and look at the front end and you can start looking at

the different piece of it as well.

So if you want to brand this site so that it has your business name,

your logo, you want to change it to the right colors,

you can do that all within the code and then you would just run A-Z,

A-Z, deploy or A-Z up.

Easy to deploy just purely for these cosmetic issues.

Aside up, if you need to recreate

a lot of the the Azure pieces.

Anyways, I hope that was really interesting.

We have done two separate chat bots now

and I think they are very different approaches

to the problem and we may actually build our own in the near future.

These are actually quite sophisticated even as samples and you can build

your own chat bot for with, with far fewer lines of code,

far simpler as well.

So thanks for watching.

And next week we will do yet another project.

I'll see you next week. Bye.

Oops.

Let me pop in really quickly with one final note here.

I was looking at the fake news for this,

some very, very useful information.

If you have additional documents you want to put in here,

you can just run a separate command as well.

So the idea is you keep this in one data folder,

you remove and you put in new documents as as you need.

The other piece other than GPT four

is there are two tabs and you can ask questions

about your document and you can also chat with them.

There are

specific differences in how it approaches.

Now if you chat,

it turns a question into a search query.

It's almost like Bing Chat if you have used it

before, you ask it a question.

Let's say what is in my health plan.

It doesn't search for that specific question.

It will turn it into health plan contents.

Let's say, and search that.

That is one way of doing it.

Another way is

the ask tab.

So it will search the question

as is and combine the search result and the user question.

I think these are two distinct different ways

to approach it, both with different kind of results.

If you have chat and it reduces your question into a search query,

it will probably get to the document,

the right document, quicker and better.

So if you have somebody folks that aren't sure what

they're asking, that's probably your best bet.

The chat tab, if they're asking questions and they know what they're

talking about, this is your best bet.

So for example, if I have a business and I have all of the policies,

internal policies of the business in this folder,

the person that uses chat is your general employee

who has no idea what they're doing and they're just saying,

I want to take a sick day. What do I do?

The ask a question would be more tailored to somebody who is writing that policy,

who knows exactly what it is they're looking for.

So they're going, you know, in their mind, page 14 of this document,

I think, and they'll know the right keywords to say,

I wrote a document that has a header that is talking about

medical sick days.

That's not even the right term.

But let's say medical leave,

your regular person might not use that terminology.

So very cool that there are two separate two separate approaches

and and their use case differs.

That's it for me.

Lots of interesting information if you read to the whole GitHub, read me.