Using Mistral Large 2 in IBM watsonx.ai flows engine
Summary
TLDRIn this video, the presenter explores using Watson's Exploratory (Mr. Large 2) engine with large language models. Mr. Large 2 excels in reasoning and code generation, supporting a multitude of languages, including character-based ones like Chinese and Japanese. The video demonstrates creating text completion and chat flows using Watson's X Flows engine, deploying them via CLI, and interacting with them through JavaScript SDK. It showcases Mr. Large's capabilities in multilingual translation and maintaining conversation context, highlighting the ease of building applications with Watson's tools.
Takeaways
- đ The video explores integrating the Mistral Large 2 language model with the Watson Explorer engine for multilingual AI applications.
- đŹ Mistral Large 2 is designed to be multilingual, supporting a wide range of languages including character-based languages like Chinese, Japanese, and Hindi.
- đĄ The model excels in reasoning and code generation, making it a valuable tool for developers looking to build applications with diverse linguistic capabilities.
- đ The Watson Explorer engine is introduced as a framework for building AI flows, which can be used for various AI tasks like text completion, summarization, classification, and more.
- đ ïž The video demonstrates creating a text completion flow using the Watson Explorer engine, showcasing the ease of deploying AI flows to a live endpoint.
- đ The Watson Explorer CLI is used to deploy the flow, and an SDK is available for JavaScript and Python to interact with the deployed flows.
- đ The video provides an example of using Mistral Large 2 for text completion in multiple languages, showcasing its capabilities with character-based languages.
- đŹ A chat flow is also demonstrated, where the model maintains context across multiple user inputs, simulating a conversational interface.
- đ§ The video suggests that for practical applications, maintaining state across interactions would be necessary, hinting at the need for client-side or server-side solutions.
- đ The script mentions future content on tool calling with Mistral Large, indicating ongoing development and exploration of the model's capabilities.
Q & A
What is the main focus of the video series?
-The main focus of the video series is exploring how to use the Watson Explorer engine with different large language models, including IBM Granite and MeLLa 3.1, and in the final video, with Mistral Large 2.
What are the capabilities of Mistral Large 2 according to the video?
-Mistral Large 2 is capable of reasoning and code generation. It is multilingual by design, supporting not only languages like English, Spanish, and French but also character-based languages such as Chinese, Japanese, and Hindi. It also excels at reasoning involving larger pieces of text due to its large context window of almost 130,000 tokens.
What is Watson Explorer Flows engine?
-Watson Explorer Flows engine is a framework for building AI flows. It allows users to use a declarative flow language to build any AI flow ranging from text completion, summarization, classification, to retrieval augmentation, and generation.
How does the Watson Explorer Flows engine work with the CLI?
-The Watson Explorer Flows engine uses the CLI to deploy flows to a live endpoint. Users can deploy their flow configurations and environment variables by running the 'WX flow deploy' command.
What is the purpose of the JavaScript code example in the video?
-The JavaScript code example demonstrates how to interact with the deployed flows using the SDK. It shows how to set up the Watson Explorer endpoint and API key, and how to send requests to the flow, such as asking for translations in different languages.
What is the significance of the model variable in the text completion flow?
-The model variable in the text completion flow allows the user to change the language model being used without modifying the flow itself. Although the video only uses Mistral Large, this flexibility enables the use of different models in the future.
How does the video demonstrate the multilingual capability of Mistral Large 2?
-The video demonstrates the multilingual capability of Mistral Large 2 by asking it to translate the word 'computer' into five different languages, showcasing its ability to handle both alphabetic and character-based languages.
What is the purpose of the chat flow example in the video?
-The chat flow example in the video is to show how Mistral Large 2 can maintain context over multiple turns of a conversation. It uses a chat flow to answer questions and provide follow-up responses based on the chat history.
What is the role of the 's' and 'instruct' tags in the prompt template?
-The 's' and 'instruct' tags in the prompt template are used to help the language model parse the prompt more effectively. They provide instructions and context to the model, which aids in generating more accurate responses.
How can the state be maintained in a chat flow as demonstrated in the video?
-In the video, the state is maintained by sending the entire chat history with each request. However, for a more practical application, it suggests using client-side JavaScript or server-side routes to store and manage the state between requests.
What is tool calling and when can we expect to see a video on it?
-Tool calling is a feature that allows the language model to interact with external tools or services. The video mentions that a video demonstrating tool calling with Mistral Large will be released soon.
Outlines
đ» Introduction to Working with Mistral Large 2 and Watson X Flows
This paragraph introduces the final video in a series focused on exploring the use of Mistral Large 2, a language model adept at reasoning and code generation, with Watson X Flows engine. Mistral Large 2 supports a multitude of languages, including character-based ones like Chinese, Japanese, and Hindi, making it suitable for developers working with diverse linguistic backgrounds. The video demonstrates how to deploy a text completion flow using Watson X Flows engine and Mistral Large 2 through the command line interface (CLI) and interact with it using JavaScript SDK. The flow is designed to be adaptable, allowing for changes in parameters such as temperature and stop sequences. The deployment process is showcased, highlighting the ease and speed with which a live endpoint can be created.
đ Deploying and Interacting with Text Completion and Chat Flows
In this paragraph, the video script details the process of deploying a text completion flow using the Watson X Flows engine and Mistral Large 2. The script provides a step-by-step guide on setting up a JavaScript application to interact with the deployed flow. It showcases an example where the model is asked to translate the word 'computer' into five different languages, demonstrating its multilingual capabilities. The video then transitions to creating a chat flow, which requires maintaining the chat history for context. The script explains how to set up the chat flow in the Watson X Flows engine, deploy it, and interact with it using a JavaScript application. It also touches on the concept of tool calling, which will be covered in future videos, and encourages viewers to subscribe to the YouTube channel for regular updates.
đș Conclusion and Invitation to Explore More Watson X Flows Applications
The final paragraph of the script wraps up the video by inviting viewers to explore other videos in the series on the YouTube playlist. It emphasizes the variety of applications that can be built using Watson X Flows, such as summarization and RAG (Retrieval-Augmented Generation). The script encourages viewers to subscribe to the channel and like the video to stay updated with the weekly releases, promising more content on tool calling and other advanced features in future videos.
Mindmap
Keywords
đĄExplow Engine
đĄLarge Language Models (LLMs)
đĄMisto Large 2
đĄMultilingual Support
đĄContext Window
đĄText Completion
đĄChat Flow
đĄCLI (Command Line Interface)
đĄSDK (Software Development Kit)
đĄPrompt Template
Highlights
Introduction to using an explainable engine with large language models, focusing on IBM Granite and MeLLa 3.1.
Demonstration of working with Mistral Large 2, known for its reasoning and code generation capabilities.
Mistral Large 2's multilingual support, including character-based languages like Chinese, Japanese, and Hindi.
Mistral Large 2's large context window of 130,000 tokens, beneficial for reasoning with large text pieces.
Using the Watson X Flows engine to build AI flows with a declarative flow language.
Deployment of a text completion flow using the Watson X Flows engine and CLI.
Interaction with the deployed flow using JavaScript SDK to perform text completion.
Translation of the word 'computer' into five different languages using Mistral Large.
Setting up a chat flow in the Watson X Flows engine for conversational AI.
Maintaining state in chat flows by sending the entire chat history with each request.
Creating a chat flow that uses Mistral Large for conversational responses.
Using the SDK to interact with the chat flow and receive conversational responses.
Handling follow-up questions in the chat flow by updating the prompt with previous interactions.
Potential for building client-side or server-side applications to maintain chat state.
Upcoming video on tool calling with Mistral Large, expanding the capabilities of the language model.
Encouragement to subscribe to the YouTube channel for weekly updates on AI and language model applications.
Transcripts
welcome to a final video in a series
where I'll be exploring how to use
what's an explow engine with different
large language models so far we've
experimented with IBM granite and also
me llama 3.1 in this final video I'll be
showing you how to work with mistol
large so Mr Large 2 is really good at
reasoning or helping you with code
generation also it's multilingual by
design it doesn't only support languages
such as English or Spanish or French it
also supports a large number of
character-based languages such as
Chinese Japanese or Hindi so this will
help developers with all sorts of
different languages to build
applications and services for all sorts
of linguistic backgrounds r large 2 also
works really well when you're trying to
do reasoning also reasoning that
involves larger pieces of text because
it has a really large context window of
almost 130,000 tokens
in this video we'll be using the Mr
Large model together with what's an XF
flows engine if you're not familiar with
what's an X flows engine it's a
framework to build AI flows meaning you
can use a declarative flow language to
build any AI flow ranging from text
completion summarization classification
to even retrieval augment to generation
or rack and we have a ton of videos on
these topics right here on this YouTube
channel what we'll be doing today is
we'll be using a couple of flows mostly
vertex completion but also a chat flow
together with Mr Large and then we'll be
using what's an explow engine using the
CLI to deploy this to a live endpoint
and we'll be using the SDK to interact
with it from a little bit of JavaScript
code there is also a python SDK
available in case you prefer writing
python so let's dive into vs code and
get
started in order to create a new whatson
X flows engine project you need to
install the CLI I've already installed
the wxf flow CLI but going to the
dashboard and signing up for a free
account it's important to remember that
you can sign up for free there is no
credit card required and signing up for
free allows you to use all sorts of llms
including Mr Large which we'll be doing
in this video I've created my very first
flow which is a flow for text completion
so this flow is composed of only two
steps there's one step which is setting
the prom template and then there's
another step that's setting the
completion that goes to the large
language model in completion I've
decided to make the model a variable
that I can tweak even though we only
will be using Mr Large today I'm only
sending off a set of parameters as well
so this could be the temperature or a
stop sequence in my prom template I've
created the prom template that works
best for Mr Large as you can see here
we're setting a couple of tags such as
the S and instruct and this will help
the LM to parse the prompt we send over
in order to deploy this the only thing I
need to do is run your command WX flow
deploy and what this will do it will
take my flow configuration it will take
any environment variables I have in this
environment file and then it deploys
them to a live endpoint as you can see
here it only took 6 seconds to deploy
this text completion flow to a live
endpoint and whenever it's available on
the endpoint I can then use the SDK for
JavaScript or python to interact with my
flows I've set up a small example
JavaScript code um in the beginning you
need to set up your WX flows endpoint
and API key and of course I've hidden
this information for you which you
should uncommon these lines and add your
own values um and then finally I'm going
to interact to my flow you can see I
have a function model. flow which takes
these schaps it is an overview of all
the different flows I have and the SDK
will help you to generate this then I
need my flow name which in this case is
text completion I need to set some
variables and then I want to set the
model so the model I'll be using is the
Mr Large model and this is available on
what's a next and you can use it for
free in flows
engine so let's say I have a question
and remember this model is very good in
working in different languages so I can
for example ask can you
translate the word
computer into five different
languages so this should be doable for
most llms let's see what Mr Large comes
up with but first I'm going to give it a
format or a structure that I want my
answers to be in use the
following format and for this I want to
do language and then I want to do the
translation so let me save this and then
I can run the node.js app by running
note in next.js and it should return the
response in my
terminal if you look at the response you
can see it's giving me the translations
in five different languages uh Spanish
French German Italian and then also in
Japanese and a Japanese one is
interesting for this model because not
all LMS are able to work with character
only languages such as
Japanese besides Simple Text completion
we can also create a chat flow and to
implement a chat flow you normally also
need some sort of JavaScript application
or a way to main stain state for now
we're going to assume that all these
flows are text in or text out meaning
that on every request I'm going to send
over the entire chat history so let's
get back into vs code and set up a chat
flow in my WX flows ATO file I can now
create another
flow I'm I'm going to call this flow
chat and it takes two steps one step is
templated prompt which will take my
promp template as you can see this time
I'm leaving over the formatting of the
prom template to my JavaScript
application also I'm hardcoding the
model to be myal a/ mistal D large and
this is the model ID that I need to use
if I want to access myal large through
the Watson X platform if I save this and
deploy this by typing WX flows deploy in
my terminal it will make this flow
available on a live handpoint
as we saw before this will only take a
couple of seconds to deploy this
directly to the cloud in my application
I already set up the flow name to be
chat and then I then I pasted a part of
my prom template in there so in here I
can type a question
like I want to
learn more about
LMS please EXP explain it to
me
like I'm a
5-year-old and by Saving this I can now
run my nodejs app by running node
index.js and this will take my new flow
called chat it will pass in this promt
and then it will use the Mr Large model
I can actually delete this as I already
defined the model directly in my
flow if I look at the end answer it says
sure imagine playing a game blah blah
assume you want to ask a follow-up
question what I should do then is I can
copy this the response and put it in my
prompt and what I can do then I can
close this interaction by using this tag
which is
s and I can create a new instruction so
the new instruction would be the
follow-up
question like
mention playing a game where you have to
guess what someone is thinking I can
then ask give me an example of a
game and
by closing this I close the
instruction I can save it and then I
need to make sure there's a space here
so the LM knows it needs to do
completion if I save this and run my no
GS app again what it will do it will
take the entire chat history which is my
first question the
answer of the LM and then my follow-up
question and it should generate another
answer so it says let's play a game what
am I and then finally in the end it says
is it the lion yes it is it isn't
related to the LM so let's find out
what's going wrong I should probably be
a bit more explicit and say give me an
example of a game that
explains what a
large language model is and if I save
this and we my nodejs up again it should
now take the previous history and use
this to give me an example of a game
instead of coming up with a new
game and as you can see they're uh
proposing a game where uh you need to
reason and then figure out the L LM is
something that
thinks as a followup what you probably
want to do is turn this into some client
side JavaScript application or maybe you
want to build a server side route where
you're maintaining this state because
now you need to copy paste it and every
time you need to rerun uh this specific
function you probably want to store this
state somewhere and then have the SDK uh
returning it for you on every
request what you can also do with the Mr
Large models is tool calling I didn't
show any tool calling sample today but I
promise you you can find a video on tool
calling on this channel very
soon and that's how easy it is to build
a new application using whatson explos
engine together with a Mr Large model if
you missed any of the other videos in
this series make sure to look at our
YouTube playlist because there are more
videos in this series and we also have
examples on building other types of
applications such as summarization or
rag keep an eye out for our YouTube
channel because we will be releasing
videos on a weekly basis so make sure to
subscribe and like this video
5.0 / 5 (0 votes)