Using Mistral Large 2 in IBM watsonx.ai flows engine

Summary

In this video, the presenter explores using Watson's Exploratory (Mr. Large 2) engine with large language models. Mr. Large 2 excels in reasoning and code generation, supporting a multitude of languages, including character-based ones like Chinese and Japanese. The video demonstrates creating text completion and chat flows using Watson's X Flows engine, deploying them via CLI, and interacting with them through JavaScript SDK. It showcases Mr. Large's capabilities in multilingual translation and maintaining conversation context, highlighting the ease of building applications with Watson's tools.

Takeaways

• 🌐 The video explores integrating the Mistral Large 2 language model with the Watson Explorer engine for multilingual AI applications.
• 💬 Mistral Large 2 is designed to be multilingual, supporting a wide range of languages including character-based languages like Chinese, Japanese, and Hindi.
• 💡 The model excels in reasoning and code generation, making it a valuable tool for developers looking to build applications with diverse linguistic capabilities.
• 🔗 The Watson Explorer engine is introduced as a framework for building AI flows, which can be used for various AI tasks like text completion, summarization, classification, and more.
• 🛠️ The video demonstrates creating a text completion flow using the Watson Explorer engine, showcasing the ease of deploying AI flows to a live endpoint.
• 🔑 The Watson Explorer CLI is used to deploy the flow, and an SDK is available for JavaScript and Python to interact with the deployed flows.
• 🌐 The video provides an example of using Mistral Large 2 for text completion in multiple languages, showcasing its capabilities with character-based languages.
• 💬 A chat flow is also demonstrated, where the model maintains context across multiple user inputs, simulating a conversational interface.
• 🔧 The video suggests that for practical applications, maintaining state across interactions would be necessary, hinting at the need for client-side or server-side solutions.
• 🔄 The script mentions future content on tool calling with Mistral Large, indicating ongoing development and exploration of the model's capabilities.

Q & A

What is the main focus of the video series?

The main focus of the video series is exploring how to use the Watson Explorer engine with different large language models, including IBM Granite and MeLLa 3.1, and in the final video, with Mistral Large 2.

What are the capabilities of Mistral Large 2 according to the video?

Mistral Large 2 is capable of reasoning and code generation. It is multilingual by design, supporting not only languages like English, Spanish, and French but also character-based languages such as Chinese, Japanese, and Hindi. It also excels at reasoning involving larger pieces of text due to its large context window of almost 130,000 tokens.

What is Watson Explorer Flows engine?

Watson Explorer Flows engine is a framework for building AI flows. It allows users to use a declarative flow language to build any AI flow ranging from text completion, summarization, classification, to retrieval augmentation, and generation.

How does the Watson Explorer Flows engine work with the CLI?

The Watson Explorer Flows engine uses the CLI to deploy flows to a live endpoint. Users can deploy their flow configurations and environment variables by running the 'WX flow deploy' command.

What is the purpose of the JavaScript code example in the video?

The JavaScript code example demonstrates how to interact with the deployed flows using the SDK. It shows how to set up the Watson Explorer endpoint and API key, and how to send requests to the flow, such as asking for translations in different languages.

What is the significance of the model variable in the text completion flow?

The model variable in the text completion flow allows the user to change the language model being used without modifying the flow itself. Although the video only uses Mistral Large, this flexibility enables the use of different models in the future.

How does the video demonstrate the multilingual capability of Mistral Large 2?

The video demonstrates the multilingual capability of Mistral Large 2 by asking it to translate the word 'computer' into five different languages, showcasing its ability to handle both alphabetic and character-based languages.

What is the purpose of the chat flow example in the video?

The chat flow example in the video is to show how Mistral Large 2 can maintain context over multiple turns of a conversation. It uses a chat flow to answer questions and provide follow-up responses based on the chat history.

What is the role of the 's' and 'instruct' tags in the prompt template?

The 's' and 'instruct' tags in the prompt template are used to help the language model parse the prompt more effectively. They provide instructions and context to the model, which aids in generating more accurate responses.

How can the state be maintained in a chat flow as demonstrated in the video?

In the video, the state is maintained by sending the entire chat history with each request. However, for a more practical application, it suggests using client-side JavaScript or server-side routes to store and manage the state between requests.

What is tool calling and when can we expect to see a video on it?

Tool calling is a feature that allows the language model to interact with external tools or services. The video mentions that a video demonstrating tool calling with Mistral Large will be released soon.

Outlines

00:00

💻 Introduction to Working with Mistral Large 2 and Watson X Flows

This paragraph introduces the final video in a series focused on exploring the use of Mistral Large 2, a language model adept at reasoning and code generation, with Watson X Flows engine. Mistral Large 2 supports a multitude of languages, including character-based ones like Chinese, Japanese, and Hindi, making it suitable for developers working with diverse linguistic backgrounds. The video demonstrates how to deploy a text completion flow using Watson X Flows engine and Mistral Large 2 through the command line interface (CLI) and interact with it using JavaScript SDK. The flow is designed to be adaptable, allowing for changes in parameters such as temperature and stop sequences. The deployment process is showcased, highlighting the ease and speed with which a live endpoint can be created.

05:03

🌐 Deploying and Interacting with Text Completion and Chat Flows

In this paragraph, the video script details the process of deploying a text completion flow using the Watson X Flows engine and Mistral Large 2. The script provides a step-by-step guide on setting up a JavaScript application to interact with the deployed flow. It showcases an example where the model is asked to translate the word 'computer' into five different languages, demonstrating its multilingual capabilities. The video then transitions to creating a chat flow, which requires maintaining the chat history for context. The script explains how to set up the chat flow in the Watson X Flows engine, deploy it, and interact with it using a JavaScript application. It also touches on the concept of tool calling, which will be covered in future videos, and encourages viewers to subscribe to the YouTube channel for regular updates.

10:03

📺 Conclusion and Invitation to Explore More Watson X Flows Applications

The final paragraph of the script wraps up the video by inviting viewers to explore other videos in the series on the YouTube playlist. It emphasizes the variety of applications that can be built using Watson X Flows, such as summarization and RAG (Retrieval-Augmented Generation). The script encourages viewers to subscribe to the channel and like the video to stay updated with the weekly releases, promising more content on tool calling and other advanced features in future videos.

Keywords

💡Explow Engine

The 'Explow Engine' is a framework mentioned in the script for building AI flows. It allows users to create various AI functionalities using a declarative flow language. In the video's context, it is used to demonstrate how to integrate with large language models like Mr Large 2 for tasks such as text completion and chat flows. The script shows how to deploy flows using the Explow Engine's CLI and interact with them using SDKs for JavaScript or Python.

💡Large Language Models (LLMs)

Large Language Models, or LLMs, refer to advanced AI models that are capable of understanding and generating human-like text. The script specifically mentions Mr Large 2, which is highlighted for its multilingual capabilities and reasoning skills. The video explores using LLMs within the Explow Engine to perform tasks like code generation and text translation across different languages.

💡Misto Large 2

Misto Large 2 is a large language model that is showcased in the video for its proficiency in reasoning and code generation. It is designed to be multilingual, supporting a wide range of languages, which makes it a versatile tool for developers looking to build applications that cater to diverse linguistic backgrounds. The script provides examples of how Misto Large 2 can be used to translate words and engage in multilingual conversations.

💡Multilingual Support

The term 'Multilingual Support' in the script refers to the ability of Misto Large 2 to handle multiple languages, including character-based languages like Chinese, Japanese, and Hindi. This feature is crucial for global applications aiming to serve users from different linguistic backgrounds. The video demonstrates this by showing how Misto Large 2 can translate the word 'computer' into several languages.

💡Context Window

The 'Context Window' mentioned in the script is a feature of Misto Large 2, which allows it to process and reason over large pieces of text. With a context window of almost 130,000 tokens, Misto Large 2 can effectively handle complex reasoning tasks that involve extensive text analysis. This capability is showcased in the video through examples of text completion and chat flows.

💡Text Completion

Text Completion is a task where the AI model generates text based on a given prompt. In the video, text completion is one of the AI flows demonstrated using the Explow Engine and Misto Large 2. The script describes creating a flow for text completion, where the model generates text based on a prompt template and parameters like temperature and stop sequence.

💡Chat Flow

A 'Chat Flow' in the script refers to a sequence of interactions designed to simulate a conversation with an AI. The video shows how to set up a chat flow using the Explow Engine and Misto Large 2, where the model responds to user prompts in a conversational manner. The script illustrates this with an example of asking the model to explain a concept in simple terms.

💡CLI (Command Line Interface)

The 'CLI' in the script stands for Command Line Interface, which is a way to interact with the Explow Engine. The video demonstrates using the CLI to deploy AI flows to a live endpoint. This involves running commands like 'WX flow deploy' to upload the flow configuration and make it accessible online.

💡SDK (Software Development Kit)

The 'SDK' mentioned in the script refers to the Software Development Kit provided by the Explow Engine. It is used to interact with the deployed AI flows programmatically. The video shows examples of using the SDK in JavaScript to send requests to the AI flows and receive responses, as well as using it to maintain state in a chat flow.

💡Prompt Template

A 'Prompt Template' in the script is a predefined structure used to guide the AI model's responses. It includes tags and instructions that help the model understand the context and desired output. The video demonstrates creating prompt templates for Misto Large 2 to ensure that the model generates responses in the desired format, such as translations or explanations.

Highlights

Introduction to using an explainable engine with large language models, focusing on IBM Granite and MeLLa 3.1.

Demonstration of working with Mistral Large 2, known for its reasoning and code generation capabilities.

Mistral Large 2's multilingual support, including character-based languages like Chinese, Japanese, and Hindi.

Mistral Large 2's large context window of 130,000 tokens, beneficial for reasoning with large text pieces.

Using the Watson X Flows engine to build AI flows with a declarative flow language.

Deployment of a text completion flow using the Watson X Flows engine and CLI.

Interaction with the deployed flow using JavaScript SDK to perform text completion.

Translation of the word 'computer' into five different languages using Mistral Large.

Setting up a chat flow in the Watson X Flows engine for conversational AI.

Maintaining state in chat flows by sending the entire chat history with each request.

Creating a chat flow that uses Mistral Large for conversational responses.

Using the SDK to interact with the chat flow and receive conversational responses.

Handling follow-up questions in the chat flow by updating the prompt with previous interactions.

Potential for building client-side or server-side applications to maintain chat state.

Upcoming video on tool calling with Mistral Large, expanding the capabilities of the language model.

Encouragement to subscribe to the YouTube channel for weekly updates on AI and language model applications.

Transcripts

welcome to a final video in a series

where I'll be exploring how to use

what's an explow engine with different

large language models so far we've

experimented with IBM granite and also

me llama 3.1 in this final video I'll be

showing you how to work with mistol

large so Mr Large 2 is really good at

reasoning or helping you with code

generation also it's multilingual by

design it doesn't only support languages

such as English or Spanish or French it

also supports a large number of

character-based languages such as

Chinese Japanese or Hindi so this will

help developers with all sorts of

different languages to build

applications and services for all sorts

of linguistic backgrounds r large 2 also

works really well when you're trying to

do reasoning also reasoning that

involves larger pieces of text because

it has a really large context window of

almost 130,000 tokens

in this video we'll be using the Mr

Large model together with what's an XF

flows engine if you're not familiar with

what's an X flows engine it's a

framework to build AI flows meaning you

can use a declarative flow language to

build any AI flow ranging from text

completion summarization classification

to even retrieval augment to generation

or rack and we have a ton of videos on

these topics right here on this YouTube

channel what we'll be doing today is

we'll be using a couple of flows mostly

vertex completion but also a chat flow

together with Mr Large and then we'll be

using what's an explow engine using the

CLI to deploy this to a live endpoint

and we'll be using the SDK to interact

with it from a little bit of JavaScript

code there is also a python SDK

available in case you prefer writing

python so let's dive into vs code and

get

started in order to create a new whatson

X flows engine project you need to

install the CLI I've already installed

the wxf flow CLI but going to the

dashboard and signing up for a free

account it's important to remember that

you can sign up for free there is no

credit card required and signing up for

free allows you to use all sorts of llms

including Mr Large which we'll be doing

in this video I've created my very first

flow which is a flow for text completion

so this flow is composed of only two

steps there's one step which is setting

the prom template and then there's

another step that's setting the

completion that goes to the large

language model in completion I've

decided to make the model a variable

that I can tweak even though we only

will be using Mr Large today I'm only

sending off a set of parameters as well

so this could be the temperature or a

stop sequence in my prom template I've

created the prom template that works

best for Mr Large as you can see here

we're setting a couple of tags such as

the S and instruct and this will help

the LM to parse the prompt we send over

in order to deploy this the only thing I

need to do is run your command WX flow

deploy and what this will do it will

take my flow configuration it will take

any environment variables I have in this

environment file and then it deploys

them to a live endpoint as you can see

here it only took 6 seconds to deploy

this text completion flow to a live

endpoint and whenever it's available on

the endpoint I can then use the SDK for

JavaScript or python to interact with my

flows I've set up a small example

JavaScript code um in the beginning you

need to set up your WX flows endpoint

and API key and of course I've hidden

this information for you which you

should uncommon these lines and add your

own values um and then finally I'm going

to interact to my flow you can see I

have a function model. flow which takes

these schaps it is an overview of all

the different flows I have and the SDK

will help you to generate this then I

need my flow name which in this case is

text completion I need to set some

variables and then I want to set the

model so the model I'll be using is the

Mr Large model and this is available on

what's a next and you can use it for

free in flows

engine so let's say I have a question

and remember this model is very good in

working in different languages so I can

for example ask can you

translate the word

computer into five different

languages so this should be doable for

most llms let's see what Mr Large comes

up with but first I'm going to give it a

format or a structure that I want my

answers to be in use the

following format and for this I want to

do language and then I want to do the

translation so let me save this and then

I can run the node.js app by running

note in next.js and it should return the

response in my

terminal if you look at the response you

can see it's giving me the translations

in five different languages uh Spanish

French German Italian and then also in

Japanese and a Japanese one is

interesting for this model because not

all LMS are able to work with character

only languages such as

Japanese besides Simple Text completion

we can also create a chat flow and to

implement a chat flow you normally also

need some sort of JavaScript application

or a way to main stain state for now

we're going to assume that all these

flows are text in or text out meaning

that on every request I'm going to send

over the entire chat history so let's

get back into vs code and set up a chat

flow in my WX flows ATO file I can now

create another

flow I'm I'm going to call this flow

chat and it takes two steps one step is

templated prompt which will take my

promp template as you can see this time

I'm leaving over the formatting of the

prom template to my JavaScript

application also I'm hardcoding the

model to be myal a/ mistal D large and

this is the model ID that I need to use

if I want to access myal large through

the Watson X platform if I save this and

deploy this by typing WX flows deploy in

my terminal it will make this flow

available on a live handpoint

as we saw before this will only take a

couple of seconds to deploy this

directly to the cloud in my application

I already set up the flow name to be

chat and then I then I pasted a part of

my prom template in there so in here I

can type a question

like I want to

learn more about

LMS please EXP explain it to

me

like I'm a

5-year-old and by Saving this I can now

run my nodejs app by running node

index.js and this will take my new flow

called chat it will pass in this promt

and then it will use the Mr Large model

I can actually delete this as I already

defined the model directly in my

flow if I look at the end answer it says

sure imagine playing a game blah blah

assume you want to ask a follow-up

question what I should do then is I can

copy this the response and put it in my

prompt and what I can do then I can

close this interaction by using this tag

which is

s and I can create a new instruction so

the new instruction would be the

follow-up

question like

mention playing a game where you have to

guess what someone is thinking I can

then ask give me an example of a

game and

by closing this I close the

instruction I can save it and then I

need to make sure there's a space here

so the LM knows it needs to do

completion if I save this and run my no

GS app again what it will do it will

take the entire chat history which is my

first question the

answer of the LM and then my follow-up

question and it should generate another

answer so it says let's play a game what

am I and then finally in the end it says

is it the lion yes it is it isn't

related to the LM so let's find out

what's going wrong I should probably be

a bit more explicit and say give me an

example of a game that

explains what a

large language model is and if I save

this and we my nodejs up again it should

now take the previous history and use

this to give me an example of a game

instead of coming up with a new

game and as you can see they're uh

proposing a game where uh you need to

reason and then figure out the L LM is

something that

thinks as a followup what you probably

want to do is turn this into some client

side JavaScript application or maybe you

want to build a server side route where

you're maintaining this state because

now you need to copy paste it and every

time you need to rerun uh this specific

function you probably want to store this

state somewhere and then have the SDK uh

returning it for you on every

request what you can also do with the Mr

Large models is tool calling I didn't

show any tool calling sample today but I

promise you you can find a video on tool

calling on this channel very

soon and that's how easy it is to build

a new application using whatson explos

engine together with a Mr Large model if

you missed any of the other videos in

this series make sure to look at our

YouTube playlist because there are more

videos in this series and we also have

examples on building other types of

applications such as summarization or

rag keep an eye out for our YouTube

channel because we will be releasing

videos on a weekly basis so make sure to

subscribe and like this video