LLM Starter Pack: A Pragmatic Guide to Success with the Large Language Models

Summary

Amir provides a pragmatic view on using large language models. He explains common applications like writing assistance and coding, but cautions about risks like hallucination. He advises experimenting to see if models solve your problem before deployment, considering costs and flexibility. Composition and design are key - build a stack with language models as one component. Sophisticated combinations of models can provide competitive advantage. Use powerful tools like language models, but with awareness of limitations.

Takeaways

• 😊 LLMs are very useful for writing assistance, coding, querying data, etc but still imperfect
• 😵‍💫 Beware of hallucinations - LLMs can make up convincing but false information
• 😏 Evaluate if LLM solves your problem before productionizing; consider inference cost
• 🤔 Minimize hallucination risks; add guardrails like human review, fact checking etc
• 🤯 LLMs enable cool things like coding co-pilots, talking to data, writing books etc
• 😎 Experiment with public models & tools to determine if LLM meets your needs
• 🔍 For production use, optimize model latency, cost etc with MLOps, quantization etc
• ⚖️ Consider open vs closed source models based on needs like privacy, cost
• 📏 Bigger LLMs learn faster but benchmarks may not reflect production readiness
• 🛠 Be composable - use LLM as part of a stack, combine with other models

Q & A

What are some of the main applications and use cases presented for large language models?

Some of the main applications mentioned are using them as writing aids, coding co-pilots to help generate code, enabling natural language interaction with data, and using them to extract information from unstructured data sources like PDFs, videos, and audio.

What risks or downsides are discussed regarding large language models?

The main risks discussed are the tendency to hallucinate or fabricate factual information, leading to incorrect or misleading outputs. Several examples are provided of language models generating convincing but false information.

How can the risks of hallucination from large language models be mitigated?

Some ways to mitigate hallucination risks include minimizing exposure through careful prompting and design, putting guard rails in place with human oversight or fact checking components, and using large language models as just one composable component in a larger AI stack.

What considerations are mentioned regarding deployment of large language models?

Key deployment considerations cover factors like cost, latency, privacy, and flexibility needs. Additional model optimization, quantization, and hardware-software co-design can help maximize efficiency of deployed models.

When is training your own large language model recommended vs leveraging existing models?

Training your own model requires extensive data, compute budget, and specialized teams. In most cases, leveraging existing models with techniques like prompting and in-context learning can meet needs without costly training.

How can combining large language models with other AI capabilities lead to more advanced solutions?

Using large language models alongside other AI modules like specialized NER or NLP models, knowledge graphs, etc. allows creating sophisticated solutions that accentuate different strengths.

What framework is proposed for evaluating if and how to apply large language models to a problem?

The suggested framework analyzes whether large language models can actually solve the problem, if solutions could be deployed to production, flexibility needs, and risks like hallucination before deciding on best approach.

How crucial is model and prompt engineering highlighted in effectively applying large language models?

Effective prompting and model optimization techniques are emphasized as critical to maximize large language model potential while minimizing cost and latency tradeoffs.

What is the outlook given on the future potential and current maturity of large language model technology?

The technology shows great promise but is positioned as still maturing rapidly, requiring thoughtful application design and awareness of limitations in present form.

Why is a composable AI stack incorporating diverse technologies suggested over reliance on large language models alone?

Combining large language models with other specialized AI components allows accentuating different strengths to create more advanced and resilient solutions.

Outlines

00:00

😀 Introducing the speaker and topic of large language models

The paragraph introduces the speaker Amir and the topic of large language models (LLMs). It talks about familiar faces in the audience, excitement to share information on LLMs, whether they are overhyped, and how they can be used to build something useful. The talk will take a pragmatic approach.

05:02

😟 LLMs have imperfections like hallucination risk

The paragraph discusses that LLMs have risks like hallucination where they make up convincing but factually incorrect text. Research involves dealing with uncertainty and imperfection. The pragmatic approach is building something useful with LLMs while managing the risks.

10:02

😲 Examples of LLM failures and hallucination

The paragraph provides examples of public LLM failures like Meta's Galactica and Google's Bard, and lawyer's use of chatGPT. This highlights the risk of hallucination where LLMs make up convincing but false information.

15:03

🤔 Framework for deciding if and how to use LLMs

The paragraph introduces a framework for deciding if and how to use LLMs. Questions include - does the LLM solve my problem based on metrics, can it go to production, how flexible on cost and latency, and how bad would hallucinations be.

20:05

🔍 Tradeoffs between LLM size, accuracy, cost and speed

The paragraph discusses tradeoffs between LLM size, accuracy, computational cost and speed. Bigger LLMs have better few-shot learning but smaller ones can be more practical. Aim for simplicity with the smallest LLM that solves the problem.

25:07

🚀 Optimizing LLM deployment cost and latency

The paragraph covers optimizing cost and latency when deploying LLMs in production. This includes model quantization, pruning, server optimization using chips tailored for inference.

30:08

😎 LLMs now possible even on laptops

The paragraph shows exciting possibility of running LLMs even on laptops today. This enables easier prototyping and experimentation.

35:11

❔ Alternatives before deciding to train your LLM

The paragraph discusses trying in-context learning and prompting fine-tuned smaller models before deciding to train your own expensive LLM which needs massive data and compute.

Keywords

💡large language models

Large language models refer to a subset of deep learning models that are trained on massive amounts of text data to generate human-like language and text. As explained in the script, they are first pre-trained in an unsupervised way, then taught to follow instructions, before being fine-tuned for specific applications. Their key capability is natural language generation. The speaker discusses both their promise and limitations.

💡hallucination / fabulation

This refers to the tendency of large language models to 'make things up' or generate factually incorrect statements. As shown through examples in the script, they can provide convincing but false information. Understanding this limitation is critical when deciding whether and how to apply large language models.

💡in context learning

Rather than extensive fine-tuning, in context learning involves providing a large language model with a prompt that establishes the desired context and constraints for text generation. As noted in the script, effective prompting allows solving many problems without costly model retraining.

💡inference cost

Unlike training costs which occur once, inference costs are paid every time a deployed model is called to generate text or complete a task. The speaker cautions that the expense of running very large models can make applications economically unviable.

💡composability

This refers to combining large language models with other AI components rather than using them in isolation. Integration with task-specific models and establishing feedback loops with human input are noted as ways to create more robust applications.

💡hardware optimizations

Rather than simply running large models on GPU clusters, optimizations like quantization and pruning can improve latency and lower deployment costs significantly. Specialized AI accelerators also continue to evolve for efficient inference.

💡open versus closed source

The script advises weighing factors like model quality, privacy, license restrictions, and reproducibility when deciding between open source versus proprietary large language models.

💡design considerations

When building applications, the speaker emphasizes architectural choices that safeguard against potential model inaccuracies - such as interfaces for human-in-the-loop validation and integrating explanatory capabilities.

💡productization

Rather than prototype experiments, the script advocates developing large language model capabilities with the rigor required for real-world products - including scalable infrastructure and sustainable economics.

💡limits of utility

While noting promising capabilities, the speaker concludes that large language models may solve fewer problems than assumed. Systematic testing on specific use cases is advised rather than chasing benchmarks.

Highlights

Large language models can hallucinate and make up convincing but factually incorrect information

These models are useful as writing aids, coding assistants, and for talking to data

Put guardrails in place when using these models to minimize risk of failures from hallucinations

Use simple solutions like prompting and smaller foundation models where possible rather than large complex models

Test models yourself on your specific use case rather than relying only on benchmarks

Bigger models can solve problems with less data but still may not reach necessary accuracy for real applications

Quantizing and pruning can optimize large language models for efficient deployment

In-context learning and prompting can often avoid the need for expensive model training

Always consider the full product life cycle costs not just initial training costs

Build composable stacks using large language models as one component rather than a single end-to-end solution

Design solutions focused on humans, explainability, and guarding against hallucinations

Thin wrappers around existing services provide little competitive advantage

Combine large language models creatively with other AI solutions as agents

These models are powerful but be aware of their limitations in hallucination and deployment

Use pragmatically to build solutions that provide real value

Transcripts

hello everyone

um I'm Amir and

it's fantastic to see some some familiar

faces after after years I I just saw

um some some some very good friends here

I'm excited to share uh this with you

and this is the right time so what I'm

going to to talk about today is

um

large language models are they real uh

are they hyped and uh how how we can use

them what should we pay attention to if

we want to build something useful with

them

and I'm going to be very pragmatic about

it and I'll tell you what what exactly I

mean by pragmatic

um so

I'm heading up data science and Ai

adarteria and what we do is we are using

similar techniques and similar science

and engineering that I'm showing you

here to a make documentation Easy by

making any unstructured data

understandable searchable and con and we

convert unstructured data to

knowledge graphs basically

and then we take that and we enable the

users to to to use it for making

recommendations for reasoning on it and

and using it to to create documents and

to

um to do difficult things that are

difficult for humans and they don't need

to be difficult for humans in 2023.

um so

um large language models if you check

Wikipedia you see that a lot of language

models are basically

um a very wide range of models what the

community is called calling large

language models today

is a subsetter that and it is uh

basically a deep learning based uh model

for language

um that has many many parameters

millions or billions or even more

parameters and they are first

pre-trained on a huge Corpus of

unlabeled text so because we are going

to come back to this for for uh

practical reasons in in a few slides let

me walk you through the process

um this is the first step in in in

creating these models this is where you

take a huge model a huge language model

and you train it on a massive Corpus of

unlabeled text in an unsupervised way

then

and and and this is a generative model

when we say large language models today

in almost all the talks these are

generative models so they learn to

predict the next word and then so you

pre-train it then you teach them to use

to follow instructions and then you

align them with with your values

and and then this this is what you use

and and you can train them to follow

instructions or you can train them to be

chat dialogues and uh and and things

like that and then

when you use it when we talk about

further fine-tuning this is when you

take all of this and then you uh

fine-tune it on a

smaller data set in a supervised way to

to make it useful for your domain

your purpose and build something that

you want better

um

there's a lot of

um chat going on in the community and

there are two camps some people think

that this is this is a total height it's

very expensive you should avoid it

because it hallucinates it it makes

things up and uh and on the other side

there are people who believe that this

is it we have the final solution and NLP

is done machines are going to take over

and and we should just sit there and use

them and and uh and that's it and

there's going to be no

no scientists in the future there's

going to be no no programmers in the

future but what is our approach

um what I believe is that we are

somewhere in the middle uh these things

are pretty real you can do you can use

them for very useful applications and

you will see uh those applications in

the talks that we have today and and

they are very powerful but we are in an

active area of research and uh if you're

not familiar with research and how how

research is done and how to deal with uh

with imperfection and how to deal with

the uncertainty that is inherent to uh

to doing research

then you would expose yourself to a huge

risk because you read a paper before

it's peer-reviewed and you think this is

the thing and you pick it and um and and

then you expose yourself to uh massive

failures

our approach is going to be totally

pragmatic we say all right these we know

they are imperfect but can we build

something useful with them and how and

what should we pay attention to if uh if

we want to use them

um and this is not about research

directions

if you want to do research you should uh

completely

um go in an orthogonal Direction here

and take risk and and that that is a

completely different story and that's

not what we're going to talk about all

right so let's first look at some some

amazing success stories of of these

large language models

uh you can use them as

um as writing eight they can be your

assistance in writing they will help you

write correctly but they will not

help you write accurately

and to know more about it see amir's

heart at noon today he will walk you

through using it to to self write a book

a beautiful uh thing and I think this is

one of the main applications of these

tools and

um I personally am using them a lot for

uh for this purpose

another one is you you can use them as

your coding co-pilot they can generate

code and they're very good at it and the

reason is

programs are structured and um and and

you can learn it and they are well

formatted and you can build uh tools

that help you write better code much

faster

again this is a co-pilot it's not

um it's not an autopilot and if you use

it correctly for those who have been

using it correctly this enables you to

do a lot in a in a much shorter time

correct

uh this one if you go to to Aladdin's

talk uh next this is the next talk

coming he will walk you through

um all of it and we'll show you how to

use agents and

um how how you can use large language

models to code and uh and and he will

talk about a lot more as well

then

you

can use them this is kind of a future

but it's actually today as well you can

use them to change the way you use your

data and you talk to your data you can

actually talk to your data you can use

your own language

um for for programming

and

um for example databricks release their

Pi spark AI package recently and uh and

you can see that you can say what you

want to do with your database and and

then it goes and generates code and and

creates agents and it it use it does for

you what's what you need to do

um another one is pandas AI where you

can you can use it to talk to your

pandas in Python uh very interesting and

and Gabriel is going to to talk about

this uh later today

another very interesting application is

you can chat with your data with your

unstructured data in whatever structured

or unstructured in whatever format they

are and this is a very interesting

application

um you can apply to PDFs to YouTube

videos to audio to name it whatever you

can imagine your data to be in you can

you can use it to talk to it and for

example here's an example of

um asking

uh what is what is important about this

paper and uh and you get the result and

um Dennis is going to talk more about

this aspect of stuff today

and you will learn a lot more

interesting aspects of this

now so that's fantastic right it it

looks like we have managed to build

something that is completely automated

with very little effort what I showed

you you can you can do with very little

effort if you use the right model and

the direct framework and it looks like

if you have seen the modern times it

looks like the beginning of the scene

where they introduce the machine

completely automated but then if you

have been reading the news you see

things like this that

meta releases Galactica and then

it has to uh take it down after only

three days

well

then then Google releases Bard and then

the the shares fall

and more recently lawyers use chat GPT

and then

um interesting stuff happens and and

they they have to defend themselves

um and now

so suddenly it looks like the thing the

machine that we thought was working

perfectly is uh is not perfect and the

question is

what happened and uh what is what is

going on with this

I want to talk about this the first

thing and then

um hopefully everyone uh will uh we will

agree quickly and and then we can move

on to the to the next

um stuff

what happened here is called

hallucination and apparently there is a

better word for it uh called fabulation

which is a more psychological term

um these models are prone to making

things up

and because these are Auto regressive

models they

they generate

sequences that sound per that sound like

perfect English but there is but they

are factually wrong

um this is my experiment I asked Chad

GPT whether Einstein could have seen the

could have heard the news about landing

on the moon and when you read it in the

first read it it sounds perfect but then

you are like wait a second what and then

you you read it again and you see that

this is actually

um factually wrong

well

maybe it's because they don't have

access to to the internet what if we

give them access to the internet so I I

used you.com

and I asked about myself the model knows

about me iranian-american astrophysicist

okay close that's that's good work done

CMB but gets everything else wrong so

where I worked and what I did and it

thinks that I I want a grouper prize in

cosmology in 2018 I wish it was true but

unfortunately it's not so

um these

might have fixed by now if you if you

try it again it may not give you the

same answers but that's not the point

what I want to show you is that these

models can make things up and this is a

known fact and it's because of the

nature of these models and we should be

aware of it and when we build something

we should take that into account

all right so

the first rule is if you're going to use

it make sure that you minimize your uh

exposure to the impact of hallucination

and there are ways of doing that this is

an area of research is very active and

there are tools and ways of doing it we

don't have time to go into it but you

should be aware of that

if you compare it with self-driving cars

uh we are around here so we have things

that work and they are very powerful and

very promising and you can you can

actually use them but you cannot put it

on autopilot completely or they will

drive you into a plane and and they will

you know you you will be

um damaged

and um the the right way of using them

is to make sure that you put God razor

on them and you put humans somewhere or

you put fact checking or again uh other

other ways but we're around here and

it's very promising

all right now instead of talking about

rules and do this and uh you know you're

missing out if you're not doing these 15

things and uh uh useless conversations

like that let's build a framework of

thinking that would help us uh think

about large language models and and

figure out what to do when and how

foreign

so the first question is do I really

need an a large language model to solve

my problem

um well the first thing to to try and to

be very honest with yourself about is

a large language model actually solving

my problem and by solving we mean what

is what is your desired metric

take it and run it on your task

experiment with it this is a machine

learning solution and we know how we how

how how to use machine learning

Solutions

so take it and um and and do an honest

statistical

um benchmarking on on your use case and

see if it if it solves your problem

and uh when you do it when I say use it

I mean take it use it zero shot orbit in

context learning and then if you have to

do fine tuning then then think about it

and and ask yourself do I really have to

do fine tuning or can I do better

prompting and in context learning to

um to to get the best out of what I have

the second one is okay if it solves my

problem can I really take it to

production

and we will talk about taking things to

production because these are massive

beasts these are these are huge and

there are considerations when you think

about taking these to production then

think about how flexible you are how

much money you have okay and how much

money you're willing to spend on the

cost of inference let's say you're

you're not training or fine-tuning but

the inference cost could be very high

and and you should be aware of it and

you should

you should be you should have a good

idea about what you want to spend on and

uh how flexible are you about latency

are you are you going to use them for

sub Second Use cases well that's a

different story or can you if you're if

you're writing a book with it you will

be fine waiting for even five minutes

for it to to to generate stuff for you

um and how bad would it be if it starts

making things up do you have their guard

rails in place or are you exposing

yourself to huge uh risk of failure and

um now how to answer these questions is

actually very easy we are in 2023 thanks

to hugging face

you have access to a lot of these large

language models the open source models

are available to you you can uh you can

go there call them use their uh use them

to to solve your problem and experiment

with them

um and you don't need anything else

basically you just you just need to know

how to how to load them and and use them

for uh zero shot or in context learning

uh something very useful that I

recommend to everyone is GPT for all

this is a tool that

um I have it on my machine I use it a

lot and and also you can keep install it

and use it for a lot of your prototyping

and experiments

and um all the Enterprise solutions they

have playgrounds they give you free

access to uh to their stuff so you can

go there and and experiment with it and

and you know quickly get to the bottom

of this question do does it does it

solve my problem yes or no and uh and

then make a decision and then also on

this side ask yourself

can I solve it with a smaller Foundation

model you know because Foundation

Foundation models are you have a huge

family of foundation models and you know

our own bed and and all of its

descendants and everything else that you

have there you have very powerful model

there and a lot of solution a lot of

problems you can actually solve with

with zero shot if you know how to how to

work with this stuff or you can build

few short Learners and and use them to

solve your problem

or you can fine tune them and you can

fine tune a much smaller models of

magnitude smaller and cheaper but then

you have to pay Upfront for fine tuning

and then you do your calculation and you

decide what is right and and what

actually works for you

um here

my recommendation is Occam's Razer and

think about the simplest solution that

is going to solve your problem remember

you have to you have to send these to

production and you have to maintain them

and

um and and picking simple goes a long

way

yeah

now how do you pick it

do should I use open source or closed

Source I look at all the benchmarks

and I get confused and and all of that

but

um what is important is again

experimenting and figuring out what

actually solves your problem I would say

ignore all those benchmarks if you can

and do a little

um statistical testing for yourself

because you know what problem you want

to solve

there are models that are

not on that leaderboard they are not

high on that leaderboard but they

actually can solve your problem well and

they are very small and and you can use

them

um how important is privacy to you and

ownership of your model can you send

your data to a third party or not do you

want your results to be reproducible and

how much you want to pay for it uh these

are important factors that would tell

you whether you can

use closed source

models or or whether you need to take

open source and make it your own and

deploy it in your environment to use it

and then if you if you decide you want

to use open source large language models

then make sure you understand the

license

uh this is a bit tricky it's open source

but it doesn't mean that you can use it

for business and uh make sure you

understand that and pay attention to the

quality of the results see what solves

your problem

pay attention to the size because size

bigger size means higher inference cost

and

um and and higher latency

is going to talk about the whole

landscape and he is going to walk us

through how to think about it and how to

pick and uh what to do there

all right so I should be using the

largest model right and I will be

missing out if I if I don't do it

actually uh site matters and there is a

very interesting paper that

um I recommend reading shows you how

size affects your your result basically

the

essence of it is bigger models or better

Learners and they can do better with

fewer data so you can they begin models

are better few shot learners

and uh that means if you use bigger

models it's more likely that you can

solve your problem with less data but

when what we're talking about here if

you look at this uh this core this is

the super glue score on

on some large language models we are

talking about 70 percent

score

if you are in this business you know

that you cannot sell 70 percent

um if your F score is 70 uh your clients

are not going to pay for it okay that's

why you put a lot of effort in to to go

that last Mile and push it and fine-tune

your models and get to the around 90

percent Mark where you can actually

compete with humans and you can do

something that that people are willing

to pay for so

don't forget that it's important

um with these models you can build

things that kind of work

and

uh and and using it in production for

real stuff is a different story

um a a very interesting uh case study

has been done by refuel and they looked

at labeling using Elements which is

basically a zero shot application the

the reason I'm showing it here is to

show you that look 25 is actually a tiny

model compared to these these massive uh

models and if you look at how it is

doing it's actually doing pretty good

and if you look at the cost per label

it's the cheapest

and uh in terms of latency it's not it's

not too bad

um so

think about it

and and don't go after the biggest most

expensive slowest ones and again asked

what actually solves your problem and

again when you think about benchmarks

take them with use them with care

because first of all uh what has a high

Benchmark on the leaderboard doesn't

necessarily mean that that's the best

thing that solves your problem and

um

evaluations for large language models

we're still learning about it we are

still there is uh there is an active

area of research doing that and also if

a model is closed and they are not

telling you what they have been you uh

what what they have used to train the

model and your benchmarks are available

on the internet

um just think about it that they the The

Benchmark data might have been used in

training of these models okay so again

test figure out for yourself

when you deploy the stuff

um

you so let's say you build it now you

have a solution you want to take it to

production so you need you need to

deploy them or if you're using a third

party then you need to be able to call

the API

um so

when to to use which one

um if you want to use the third party

you need to make sure that you are fine

with with your business is fine with

sending you data to a third party

usually that's if that's a big red line

and you stop there uh for us it's a it's

a it's a noble we can we can do it for

example at arterio

um so then what you need is to have to

build your own mlops team and then you

build a and and you optimize your

pipeline and you optimize your models uh

the second one is very important taking

a model in its raw form and deploying it

uh is is fine but it's not the best you

can do you can get much better latency

much better cost

if you do deep learning engineering on

your models and quantize them and plune

them and use diversity and

um gglm there's a lot of uh there's a

lot you can do there and there are good

um there are good ways of using and

deploying these models uh hugging face

inference databricks base 10 gives you

out of the box easy

um deployment Solutions and AWS is

coming up with chips that are specific

for for inference and and they are

um they are optimized for that and Intel

is releasing very interesting stuff

um to combine hardware and software for

for optimized inference

um RB and financial are going to talk

more about different aspects of this

later in the in the afternoon so don't

miss it

all right and here is something very

interesting that I was impressed with I

thought you would enjoy seeing as well

uh here is me uh running a large

language model on my laptop so if you

think when you when you use a large

language model you need massive gpus and

distributed and cloud and all of that

that's that's not the case you can

actually uh use them on your laptop

today and uh and that's that's huge it's

great for prototyping it's great for a

lot of experiments and for having a

little assistant next to you this is

possible thanks to ggml

all right

do you need

to train your model your own large

language model or

can you build useful stuff with the with

the existing models

if you want to train your own mother

think about it ask yourself is it really

necessary or am I thinking about

training because I'm used to doing

machine learning that way and and I

think if I want to do something of my

own I have to train my model

and the reason you should think really

hard about it is because these models

are massive and in addition to a lot of

compute a lot of GPU TPU time you also

need a lot of data to be able to build

something meaningful there so ask

yourself do I have the data do I have

the budget do I have the time do I have

the right team and do I really have do I

really need to to train my own or or can

I

use in context learning with good

prompting or maybe a little bit of

fine-tuning

and uh and and think about

when you it's not when you train it's

not a one-time thing you have to train

and retrain and retrain and uh and and

the cost uh quickly adds up and think

about what actually you you gain if you

if you do that

um

so here I wanted to say

um again most of your problems most of

the problems that I have seen you should

be able to do with uh within context

learning and um and you know using

something like ESP or or something

similar is going to

solve a lot of your problems and then

your large language model could be one

component in your whole stack we will

talk about it in a second and

if you start getting into this phase of

no I have to fine tune and I have to

collect data and I have to do it a

supervised training to to improve the

resource and get what I want then take a

step back and look at it again and ask

yourself

um is my problem best solved with a

large language model or since I'm I'm

going to fine tune can I build something

which is way smaller and and I fine-tune

it and and I I create something that is

fast and efficient and solves my problem

at a a fraction of that cost

um

and here is my uh the next two slides

are two

um

to two recommendations that I have for

you if you if you're doing this

use the power prompting use in context

learning and push it to to its limits

and

um

make sure that that you know how to do

this efficiently that solves a lot of

your problems

and always think about your your your

stack your your Hardware

imagine

and know that that you might be able to

to do a lot of things without using the

most expensive

um Hardware

and if you

if you do a little bit of deep learning

engineering on your model and if you

think about it right you my experience

is that you're always able to find a

solution that is cheaper faster and uh

and remember inference costs are

different from training costs training

cost is a one-time thing in the

beginning

and then you're good for for a while

inference cost

you're paying it every time someone

called your API and they add up quickly

and they make it expensive and and that

cost brings your margin down and then

then you will end up building something

that works and is interesting but you're

breaking the bank and you can like not

make money with it and it's not

sustainable and your idea dies and your

product doesn't go anywhere and uh be

composable build a Tiramisu at arteria

we love to build pyramid suits because

we love kiramisu

um I think the all of these different

things as layers in your Tiramisu and

use a large language model as one

component but remember that you can add

many more layers on top of it to make

something of your own

and and make a difference and

always think of it as a product as

opposed to a jupyter notebook project

and

um

design design is very important for this

stuff make sure that you design it in a

way that protects you from hallucination

that puts the human in the center that

makes it explainable makes it

understandable and if you do it right

you will build something that will make

you very rich uh very quickly

and this is

my my final thought on this

I think

if you build a thin layer which is a

wrapper around existing Services

you will be able to build something very

quickly in hours or days and and it will

be impressive but you will not have that

competitive advantage to anyone else

because remember what you did was smart

but now that you have done it it takes

others also a few hours or a few days to

catch up with you

and then then you have you're just one

voice in

um this whole Market

think more deeply about it and start

building more sophisticated things and

um and and

one interesting use of llms

which I think is the the right so the

right question to ask today is how can I

use different AI components together to

build something that actually makes a

difference

not whether I should use LL or I should

not use elements this is this is another

flavor of machine learning models

and they can be combined with other

things you can build change with them

you can you can use them as agents and

you can combine with your let's say you

have your best named entity recognition

model that that is better than everyone

else in the world

you can use uh these large language

models as the base and then you can use

them as agents and you can have them use

your tool your model your other models

as tools and then you can see that you

quickly start building uh things that

are sophisticated and are very difficult

to to beat and compete with

and

um

so that's

it basically

um these are very powerful tools

you should use them for sure

but be aware of

uh everything that that we talked about

including hallucination and deployment

considerations and pay attention to

where you can or you should or you

should not use open versus close models

and definitely use them as writing AIDS

definitely use them as your co-pilots

for writing code and for low data

regimes and

I hope you build something

exciting and I'm looking forward to

seeing everything that might happen

after this Workshop