How to Find, Build, and Deliver GenAI Projects
Summary
TLDRThis video script offers a comprehensive guide for engineers and developers on finding, building, and delivering custom generative AI solutions. It covers tactics for project discovery, the tech stack and processes for development, and the challenges of production deployment. The speaker shares insights from his company, Data Lumina, and emphasizes the importance of prompt engineering, leveraging social media for lead generation, and maintaining applications at scale.
Takeaways
- 📚 The video outlines a comprehensive process used by Data Lumina for building custom generative AI solutions, tailored for engineers and developers interested in AI applications.
- 🛠️ The process is divided into three main parts: finding generative AI projects, building these projects, and delivering them to clients.
- 💡 When starting, it's recommended to offer general AI services and then specialize (niche down) based on client needs and feedback.
- 🔎 Finding projects involves creating a clear offer, using acquisition methods like social media and networking, and effectively selling the project through understanding client needs and building trust.
- 🤖 Building generative AI projects requires understanding the core pattern of inputs, processing, and outputs, and validating ideas locally before scaling.
- 👨🏫 Prompt engineering is crucial and should be based on proven frameworks and templates to ensure the effectiveness of AI models.
- 🛑 Minimize AI usage by using LLM processing as a last resort, and focus on maximizing functionality through regular code before resorting to AI.
- 🔧 Utilize tools like Azure Document Intelligence for tasks like document parsing, and FastAPI for web applications to streamline development.
- 🔒 Security is paramount, especially when dealing with sensitive data, and multifactor authentication should be implemented for all connections.
- 🌐 The transition from local development to production requires careful consideration of hosting options, cloud platforms, and CI/CD pipelines for consistent deployment.
- 📈 Ongoing monitoring and maintenance are essential for generative AI projects, with tools like Sentry for error tracking and Lang fuse for monitoring LLM interactions.
Q & A
What is the main focus of the video by Data Lumina?
-The video focuses on guiding engineers and developers on how to find, build, and deliver custom generative AI solutions for clients, rather than using no-code tools.
Who is the target audience for this video?
-The target audience includes engineers, developers, independent contractors, and freelancers interested in building and exploring generative AI applications.
What are the three main parts the video is divided into?
-The video is divided into three parts: finding generative AI projects, building these projects, and delivering them to clients.
What does Dave Abar recommend for someone just starting out with generative AI solutions?
-Dave Abar recommends starting general, talking to clients to understand their needs, and possibly working for free in return for testimonials and referrals to build confidence and a portfolio.
How can one leverage social media platforms for finding clients interested in generative AI?
-One can leverage social media platforms like LinkedIn and YouTube by sharing experiences, learnings, and insights related to generative AI to attract potential clients.
What is the importance of the discovery call in the sales process?
-The discovery call is important for understanding the customer's pain points, offering advice, and building trust, which are crucial for converting leads into customers.
Why is prompt engineering considered as important as coding in generative AI projects?
-Prompt engineering is as important as coding because the quality of prompts can significantly impact the output of the AI model, affecting the overall effectiveness of the application.
What is the significance of using a version control system like GitHub in managing generative AI projects?
-Using a version control system like GitHub helps in managing changes, protecting branches, ensuring security, and facilitating collaboration among team members.
What are some challenges faced when moving generative AI projects from a local machine to a production environment?
-Challenges include ensuring software reliability, dealing with hallucinations in AI outputs at scale, constant monitoring and maintenance, and managing ongoing costs.
How does Data Lumina handle the integration of AI into existing systems in their projects?
-Data Lumina uses an architecture that involves connecting with existing applications via APIs, using queuing mechanisms, data processing pipelines, and integrating outputs back into the system.
What is the role of task queuing in scaling generative AI applications?
-Task queuing, implemented using tools like Celery, helps manage the load by creating a queue for incoming requests, ensuring the system does not get overloaded and maintains performance even under high demand.
What are some best practices for monitoring and maintaining generative AI applications in production?
-Best practices include using monitoring tools like Sentry, setting up CI/CD pipelines for automated testing and deployment, implementing unit tests, and being transparent about costs associated with server and API usage.
Outlines
🛠️ Building Custom Generative AI Solutions
Dave Abar, founder of Data Lumina, introduces a video guide for engineers and developers to build and deliver custom generative AI projects. The video is divided into three parts: finding projects, building solutions, and delivering them. Dave emphasizes that the video is not about using no-code tools but about creating tailored AI applications. He shares tactics for finding projects, building with specific tech stacks, and the challenges of production deployment. The audience is encouraged to follow along with a document provided in the description.
🔍 Finding Generative AI Projects and Building Confidence
The second paragraph focuses on the initial steps for finding generative AI projects, which include defining the offer, exploring acquisition methods, and understanding sales strategies. Dave suggests starting with a general approach, gaining experience, and researching successful use cases. He also recommends offering services for free in return for testimonials and referrals to build a portfolio and confidence in delivering value with generative AI.
📞 Acquisition and Sales Strategies for AI Projects
Dave discusses tactics for acquiring leads and converting them into clients. He advises leveraging current networks, social media platforms like LinkedIn and YouTube, and Discord communities. Dave also highlights the importance of providing value over direct sales pitches on social media. For sales, he emphasizes the difficulty of converting leads and suggests having strategies and experience in place. The process involves discovery calls, creating proposals, and discussing contract terms and payment schedules.
🚀 Leveraging Freelancing in the Age of AI
In this part, Dave explores the impact of AI on freelancing, citing statistics that show an increase in productivity and opportunities for tech professionals to build solutions for startups and enterprises. He mentions that generative AI makes freelancers more likely to pursue independent careers, and he shares his personal experience transitioning from freelancing to running a full development company due to the demand for generative AI solutions.
💡 Fundamentals of Building Generative AI Projects
Dave outlines the core pattern of generative AI projects, which involves inputs, processing, and outputs. He stresses the importance of validating ideas locally, creating flow diagrams, and taking prompt engineering seriously. The paragraph also covers the significance of data quality and the use of large language models, with a preference for Azure Open AI due to data safety assurances.
🛑 Minimizing AI Usage and Utilizing Tools for AI Projects
The paragraph discusses best practices for integrating large language models into applications, advocating for the use of AI as a last resort after all other coding solutions have been exhausted. Dave recommends using Pinecone and Langchain for robust integration and emphasizes the importance of monitoring applications at scale using tools like Langfuse. He also suggests leveraging Azure Document Intelligence for parsing documents and outlines the tech stack used for building generative AI solutions.
🤖 Project Management and Tools for AI Development
Dave covers project management tools and techniques for developing AI projects, including version control with GitHub, task management with platforms like ClickUp, and the importance of automation. He highlights ClickUp's features and integrations for task management and emphasizes the need for a solid project management approach to handle complexity and ensure project success.
🌐 Delivering Generative AI Projects: Challenges and Solutions
The paragraph delves into the complexities of moving AI projects from local development to production environments. Dave talks about the challenges data scientists face in software engineering, the importance of monitoring for hallucinations in AI outputs, and the ongoing costs of maintenance. He also discusses hosting options, the use of Docker for deployment, and the need for testing, monitoring, and security measures.
🔒 Security and Example Architecture for AI Projects
In this section, Dave discusses security measures for AI projects, including the use of multifactor authentication and secure storage of API keys. He presents an example architecture diagram for a project currently in development, which includes a secure and scalable framework using Azure services, VPN for access control, and a focus on security at every layer of the application.
🏢 Conclusion and Next Steps for Generative AI Projects
The final paragraph concludes the video with a recap of the in-depth discussion on building and delivering generative AI projects within Data Lumina. Dave invites viewers to like and subscribe for more content on freelancing and working with clients. He also encourages viewers to watch the next video for further insights into client acquisition strategies.
Mindmap
Keywords
💡Generative AI
💡Custom Solutions
💡Data Lumina
💡Acquisition Methods
💡Sales
💡Prompt Engineering
💡Azure Open AI
💡APIs
💡SaaS (Software as a Service)
💡Freelancing
💡Project Management
Highlights
Dave Abar introduces a comprehensive process for building custom generative AI solutions at Data Lumina.
The video is tailored for engineers and developers interested in exploring AI applications, not for those seeking guidance on no-code tools.
A three-part structure covers finding projects, building AI solutions, and delivering them effectively.
Dave emphasizes starting generative AI projects with a general approach, understanding client needs, and then niche-ing down.
The importance of having a clear 'offer' when starting out in generative AI projects is discussed, including choosing between a general or niche approach.
Tactics for finding leads include leveraging social media, engaging in Discord communities, and participating in educational groups like Data Alchemy.
Sales strategy involves understanding the customer's pain points and building trust through expertise and valuable insights.
The video discusses the importance of prompt engineering in generative AI and the significance of data quality over the AI model used.
Dave recommends using Python for backend development, Azure Open AI for model provider, and emphasizes minimizing AI usage until necessary.
An example architecture diagram is presented for a robust and secure generative AI project integration.
The video covers the complexities of moving from a local development environment to a production environment for AI projects.
Different hosting options are explored, including client-hosting and self-hosting solutions with considerations for recurring revenue.
The use of Docker for consistent and scalable application deployment is recommended, along with setting up CI/CD pipelines.
Security best practices such as using multifactor authentication and VPN connections are highlighted for secure application development.
The video concludes with a call to action to subscribe for more insights on freelancing and AI project management.
Transcripts
how to find build and deliver generative
AI projects in this video I'm going to
walk you through the entire process that
we use inside of our company data Lumina
to build custom generative AI solutions
for our clients now on that note this
video is not for you if you're looking
to learn more about how to build and
deliver generative AI Solutions using
nood tools like sapper make voice flow
or bpress which are all great by the way
but this video is really going to dive
into building custom Solutions so this
video is for engineers and developers
who want to build and explore generative
AI applications in their own company or
for example do that as an independent
contractor or freelancer to really start
your own thing which is really a great
opportunity right now for every
developer that is really interested in
working with generative AI this video is
split up into three parts first we'll
look at how to find generative AI
projects we'll share some tactics that
we use inside of our company then how to
build where really get into the
specifics the Tex stack the the
Frameworks the processes the tools that
we use everything and then we talk about
how to deliver these generative AI
projects because it's one thing to build
something locally on your laptop that
works but bringing them putting that
into production where it can run and the
client can access it that's a whole
different story and that's the most
challenging part so that's how it is
split up quick context about me for
those of you that don't know me my name
is Dave abar I'm the founder of data
Lumina and we help organizations and
individuals to navigate the AI
Revolution and we do that by building
custom Data Solutions for the clients
that we work with and we also cultivate
technological leadership through our
educational resources that is our
YouTube channel our free community data
Alchemy which you can check out and our
paid Community data freelancer where we
help data professionals to get started
with freelancing by showing them how we
run our company how we find clients and
collaborating with them now if you want
to follow along with this document you
can click the link in the description
you'll uh see a link to this document if
you click it you can duplicate or
download it after signing up for clickup
so if that's something you want you can
do that you can also export it to
markdown to put into notion obsidian
whatever tool that you are using so with
that out of the way let's get straight
into things and start with part one how
to find generative AI projects I have a
diagram over here and a stepbystep
instructions uh let's first break it
down really at the core most top
fundamental level what do you need in
order to find these projects it's split
up into three parts you have the offer
you have acquisition methods so how to
find leads and you have sales after
finding those leads how do you actually
sell the project to them so let's start
with the offer this is really what are
you going to sell and when you're
starting out you should ask yourself the
question are you going to go general or
are you going to Niche down General
really is okay what's your problem you
talk to the to the clients they say hey
we have problem X Y and Z and you say
hey I can build that for you going Niche
is you you go to the client say you have
a problem I have the solution you go
specific you know a certain problem that
for example a niche or a market is
dealing with and you built a tailored
solution for that if you're just
starting out and that's probably you if
you're watching this video then I would
recommend start general first just talk
to people and let them tell you what it
is that they want and then figure out if
you can help them so how do you start
and for this you need to be experienced
so you need to be a developer a data
scientist machine learning engineer
software engineer data engineer and
ideally have some experience with
generative AI under your belth because
building custom Solutions is hard and
you you can start by for example solving
a problem that you are experienced this
is a great way to start something else
that you can do is research successful
gen fi use cases and I've listed some of
them here where you could really think
about the main departments that every
almost every company has so Marketing
sales support and operations these are
really for all of these departments
generative AI can be great and there are
lots of solutions so take some
inspiration from there see if there's
something that you like that could also
be a great starting point so you can
either just pick something on your own
start with that to build your own
portfolio or what you could also do very
powerful offer to work for free in
return for testimonial and some
referrals so go out to some people in
your Network a company your boss uh
clients you've worked with in the past
doesn't matter find a company that is
interested in doing something with
generative Ai and see if you can help
them out maybe do it for small fee maybe
do it for free but really the important
the the goal here really is to get that
case study and ideally a testimonial to
first and foremost also build confidence
so you know hey I can do this I can
actually deliver value to companies
using the skills and uh that I have and
these tools that is the offer what are
you going to sell get clear on that I
would recommend starting General and
then from there we get into the next
step which is acquisition so how to find
and convert leads and now if you're here
because you just want to know how to
build these projects or you want to do
something within your company then you
can kind of like skip over these steps
and skip this video a little bit uh
where we get into part two but if you're
new to this and you're looking to find
clients this is where you should start
so with acquisition here are some tips
to get started with this and I will
leave some I will link some more
resources later but high level start
with your current Network this is the
most uh this is the easiest way to start
so especially if you already have a case
study you have some experience just
reach out to people in your current
Network either via email direct messages
WhatsApp LinkedIn DMS and ask if the
solutions you've built could be
interesting for them or someone they
know the cool thing about generative AI
right now is that if implemented
correctly of course it can really add
value to any company any industry it
doesn't matter every company has
Marketing sales operations support and
gener that's really where generative AI
shines and then it's really about
getting the right use case making it
small and tangible enough to solve a
specific problem so start with your
network next to that I would recommend
leveraging social media platforms
LinkedIn YouTube are the two best uh
ways to get started I understand YouTube
is not for everyone can be a big step
but it's highly effective LinkedIn a
little bit easier to get started with
you just write a post a lot of people
then ask okay but what do I share what
do I write about the best thing that you
can do is share your experience and
share the things that you've learned
because in doing so you let people know
that you're actively working on this
you're learning uh you're learning about
this and people over time will naturally
start to reach out to you asking like
hey I'm actually interested in a you
talk about ax all the time can you help
me with this that's how it works it's
really powerful and even even like one
LinkedIn post could get the ball rolling
so don't underestimate this then another
important principle to keep in mind when
posting on social media don't sell
directly give value so don't go out on
LinkedIn for example asking all the time
like hey I'm looking for clients to work
with can you help me now occasionally
you can do this but the long-term play
is to give value so share what you've
learned share interesting insights share
knowledge share news and over time
people will naturally start to reach out
to you that's the principle that you
should keep in mind another way to find
leads is by by actually posting in
Discord communities for popular
Frameworks and tools that you're for
example working with so this is a very
interesting one and is gaining more and
more popularity but by just showing up
there and answering the technical
questions people might have they will
naturally again start to reach out to
you asking like hey I'm actually working
on this project I have this question
about this particular framework here I
give an example of Fai and then they
they come into that Community to look
for answers and if you see that they're
the expert and they're willing to pay
you some for that they might hire you as
a consultant or a developer to start
working with it interesting place for
developers to find leads then another
great way to connect with people and
share insights is via school so you can
for example come to data Alchemy which
is my free group I will leave the link
in the description uh we have over
15,000 members over here and it's
actually quite funny the people that
always show up here in the community and
that post a lot that add value people
will naturally start to reach out to
them and ask for help so for example
that happened to to Brandon uh multiple
times already because he's really active
in here and that could be a great way to
get your foot in the door as well and
then finally you could look into
freelance platforms like Fifer or upwork
but please note while these platforms
are great they can work in the beginning
it's really challenging to get started
with that all right and then let's get
into the next part which is sales so how
to sell a project and the thing that I
want you to keep in mind here is finding
leads is easy sales is hard if you don't
have the right experience or strategies
and the interesting thing here is that
everyone thinks that finding potential
customers finding leads is hard because
this is the number one question that uh
people ask when for example they come to
us on a discovery call for data
freelancer number one question that they
have is how do I find clients I always
tell them finding leads finding clients
is very easy you can literally walk up
to any business owner right now probably
and ask them hey is AI on your road map
do you want to do something with AI
probably all of them almost all of them
will say yes we're not sure but we know
AI is there we want to do something with
this that's a lead they're interested
but now converting that lead into a
customer I.E sales by now coming in
stepping in saying like here's how I can
help you here's the price here's what I
can do building confidence building
trust that is a lot harder again if you
don't have the right experience and the
right strategies so how do you do it
correctly and of course I'm going to
give you a summary here all of the
topics that we're discussing here could
have a separate 60-minute video on its
own but this video the goal is to give
you an outline if at any point in this
video by the way you have questions or
you want me to dive deeper into certain
topics leave a comment I'll consider it
for a future video or I I can I can
answer you in the comments but here's
how we do it at data Lumina currently
and for this I'm going to scroll up to
this image over here let me see yeah so
we talked about the offer we talked
about the acquisition strategies and and
by the way for our company right now
just in case you're curious all of our
clients come through either LinkedIn
YouTube or referrals or previous clients
that's where we are currently at so we
don't have to do any called Outreach
send messages that's how powerful social
media can be but now let's get into
sales so sales for us at least for most
companies starts with the discovery call
so this is an online call we do that via
Zoom we let people schedule via kly um
we also sometimes let them fill out some
questionnaires so
uh what's your company about what's the
problem what's the budget so we get some
data points already and on the call you
shouldn't really think about it as a
sales call because really while it is a
discovery call really the core thing
that you should do is ask questions lots
lots of questions so you don't open open
up the call and say hey I'm an I'm a
developer I have this I can do that I
can do that I can do that do you want to
buy from me no that's the last thing you
want to do a successful Discovery call
is asking as many questions as possible
to first of all really understand the
pain points of the customer understand
that you can actually help them
understand that it's the right fit and
in doing so you also build trust because
you show that you're an expert you ask
the right questions you trigger them
next to that you can also offer advice
the funny thing is a lot of
companies they have no idea what they
want the thing that they they come up
they they show up to you on a Q on a
discovery call and they say we want to
do things with AI sometimes it's a
little bit more specific we want to do
AI for marketing we want to do AI for
Content but they they have no idea so
you should then also step in as the
expert and offer advice this again
builds trust higher chance that the sale
will come through it's also important on
that initial call to talk about some
price expectation hey are you sure you
want to do this the scope that we're
talking about right now this could very
well go into the 15 15 20K depending of
course on the size of the project make
sure that they are aware of the the kind
of range that that J you're at for such
a project now later you can see the
arrow over here we after the call you
can create a specific proposal because
because a lot of the times you get a lot
of data on that call by asking all of
those questions and then you need to
take take a step back and dive in and
really create a thorough proposal so
this is where you really where you
outline scope timelines prices
deliverables and then you put a price on
it this can either be a fixed price so
you say okay this is a 10K project it's
going to take me four weeks this is what
I'm going to deliver you can do
sprint-based pricing where you say okay
we do two week Sprints uh price for
Sprint is X or you could say here's my
houry rate I think it's going to take me
around 40 to 50 hours maybe and we'll do
it like that there are multiple ways to
go about this and you should figure out
something that works for you what we
usually like to do is fix fix prices in
set intervals so for example we sell a
proof of concept we sell an MVP we for
example put four weeks on it it 6 weeks
on it Etc to clearly Define the scope
but talk about it on the first call so
they know what to expect also important
talk about potential hidden costs API
cost management cost hey do you know
actually if we if we implement this
solution for you do you know that we can
build this there's an upfront cost for
the development but also there's going
to be maintenance cost API costs give ex
set set the ranges for them give the
expectations so they know oh it's not
only upfront I pay you one time but this
is an ongoing thing because maintaining
and improving custom Solutions like this
a lot of work it's challenging it takes
time so you need to get paid for that as
well usually what we like to do once we
create the proposal which we usually do
either in a simple document or
PowerPoint presentation we discuss the
proposal on a second call or share it
via email we found that uh discussing it
on a second call where the client is
there works better because if you share
it via email they might have lot lot of
questions um they're not sure about
things and sometimes they just ghost you
and then next step if everything is all
right you can sign a contract so put a
contract in place uh we use doku sign
for that simple contract so let them
sign the contract and also consider a
payment schedule what you can do is you
can do for example 50/50 split 50% up
front 50% afterwards by weekly payments
uh payment uh after successful delivery
something you can play with so that's
how sales Works high level how we do it
what you need to consider all right
right and before we get into part two
there's one more thing that I want to
show you and that's this interesting
article over here thanks to AI we're in
the Golden Age of freelancing there are
a lot of interesting statistics in here
and what they found they interviewed a
bunch of top Tech professionals and they
indicated that they are all leveraging
AI first of all to increase their
productivity well this is I think what
we all should be doing but they also
capitalize on this generative AI Hye by
Building Solutions for startup and
Enterprises so they took a group of top
Tech professionals and it they it
basically said that half of them had
already built a solution a gen solution
for startups of uh or or an Enterprise
and 68% of the respondents in the survey
said that generative AI makes them more
likely to pursue an independent career
working for multiple companies it's a
really interesting article a really
interesting take out how generative a
automation reduces the need for
Freelancers in some Fields copywriters
Marketing the stuff that can really be
automated but on the other hand there
because companies notice that they can
automate this they now need Freelancers
contractors who can help them to create
these automations and this is an ideal
opportunity for for Freelancers because
these are companies that don't have ai
Engineers or data scientists inside of
their company on payroll that wouldn't
make sense they just need someone a
technical partner freelancer contractor
whatever that can help them every now
now and then to build these automations
and then they can over time replace the
overhead that they're spending on
customer support marketing Etc and now
funnel that money into potentially you
who steps in as a contractor really
interesting take and it's definitely
something that I've noticed um I've been
freelancing for 5 years already
basically whenever when uh the open AI
API came out I almost completely
switched to building generative AI
Solutions just because of how much
demand and opportun unities there is so
for my freelancing business which um is
now not even a freelancing business
anymore data Lum is now a full full
development company but I've transition
I've been able to make that transition
because of generative AI do you want to
dive deeper into how to find J fi
projects so how to find clients then you
can check out this YouTube video some
link that as well so this is how to find
freelance data and AI project in 2024
it's a 23 minute video going deeper uh
into the more specific some books some
strategies about everything that we've
discussed here all right with that said
we conclud this part one let's now get
into how to build generative AI projects
and let's start with fundamentals to
keep in mind at the core every
generative AI project follows this
pattern you have inputs you have
processing and you have outputs
generative AI generative meaning output
these models gener at stuff and they
generate stuff based on input this is
your data and your prompts and the
processing happens inside the
application so how do you bring all of
this together and there is an output
this can be in the form of text U this
can be in the form of even images this
can be in the form of any type of
combination documents you can put it
together and then also you have just the
file outputs but you also have the
destination same for the input you have
the data but you also have to Source
where it's coming from and usually what
you if you consider a full generative AI
project it combines the elements of
getting some information from a system
or an input or a form there's some type
of connection you take you take that
data you process it you add the prompts
you add the large language model you
generate something new put it into
desired format and then also put it in
uh the desired output place whether
that's on the website in a Content
management system in a on a storage
account that is generative AI projects
101 a typical flow that the applications
follow when you start to build these
projects you always want to start by
validating the ID locally meaning
getting a proof of concept you can even
sell this as a separate phase which is
what we sometimes do but you should also
be careful with that and really
communicate with the client that if you
do a proof of concept it's literally
only to test and validate f validate the
idea not something tangible that they
can use this is a mistake that that
we've made in the past where okay let's
do proof concept we're going to prove
that this works because these projects
are very Innovative Innovative they're
new a lot of these use cases are not
proven yet and sometimes there's a
little bit of ambiguity or uncertainty
whether this will actually work uh
mostly in terms of is the the quality of
the output good enough can we control
for hallucinations all of those
questions
but you want to start locally and you
should really get clear on that pattern
so get clear on the input data the
processing step the desired output in
the location and usually it helps to
create a flow diagram so create a flow
diagram you could use something like
draw.io or figma and write out the
applications flow think about where do
we need to get data from what do we need
to combine what kind of prompts do we
maybe need and then what is that output
uh going to look like then also take
prompt engineering seriously so don't
just write a prompt simply by the first
thing that comes up uh when you're
working on this or by asking C GPT to
write the prompt for you don't be lazy
here your prompt is almost as important
maybe it is as important as the code
that you're writing so follow proven uh
prompt Frameworks and templates more on
that uh in a little bit but take prompt
engineering seriously and also consider
the data is more important than the AI
that you're using with when it comes to
generative AI everyone has the same
models almost of course there's fine
tuning all of that but generally
everyone is using the same intelligence
here the difference is the data and the
data is in terms of your your literally
your input data your company data
proprietary data that that you're using
but also your prompts so combining those
two is really what makes your
application unique and potentially
better than the competition for example
all right then let's get into tools and
techniques so we use use pure python to
build our generative AI solution so we
use Python backends for prompt
engineering something that you could
look into is we like to structure our
prompts by first always focusing on a
role then a task description step by
step using Chain of Thought prompting uh
instructions we give it context we add
emotional impact and we also add some
some one uh some few shot examples and
then we tie all of this together in a
markdown format so these are proven
prompting techniques based on literature
so so if you're interested in that I
would recommend looking this up then for
large language model provider so the
brain the intelligence of your
generative AI applications uh you can
consider open AI Azure open AI or CLA we
almost always use Azure open AI because
we have the added check mark where
Microsoft ensures that they're not
sharing your data with open AI even
though Microsoft is partnered partly
owns open AI but for the clients that we
work with we can say hey your data is as
safe as the emails in your outlook inbox
um most of our the clients that we work
with they they are already using
Microsoft it's a trusted partner so that
just helps that's why we use Azure open
AI here's a big one huge one which I
also have a video on utilize penic and
instructor for robust llm integration I
will not be diving into what that is all
about but I will link this video this is
going to completely change the way you
build llm application development best
practices when it comes to large
language models and AI be Innovative but
minimize AI usage use llm processing as
a last resort what do I mean by this if
we consider the typical flow of input
processing output Etc there are
naturally going to be steps within your
application where some of these
processing steps you can just do through
a regular function maybe through regx
maybe through a simple IFL statement
make sure to
maximize that functionality first and
using an llm should really be your last
resort so really when you cannot
continue in your data processing
pipeline without AI that is where you
introduce Ai and why do you want to do
this because first of all it slows down
your application it introduce cost it
can introduce hallucination the more API
calls you have to make the more
challenging it's going to be to scale
and optimize and maintain your
application now we're talking about
generative AI application so at some
point you're going to have to ju
introduce an llm or AI because otherwise
it's not the generative AI application
but just keep that in mind because these
models are so good they can if if you
really break it down to like if you
really break problems down to the most
fundamental small part an llm can solve
almost any problem especially if you
utilize penic and instructor you can let
it analyze all kinds of problems and
then give for example a Boolean value
true false and use that to decide what
the next flow of the application should
be but really think about it can I do
this just with code first if not only
then resort to llms here's another just
random tip uh we leverage Azure document
intelligence a lot for parsing documents
so uh rack is a common use case for
generative AI
applications a problem with rack is
splitting up the data into relevant
chunks that you can then put into your
vector database for the retrieval we use
Azure document intelligence to split up
documents and let it output markdown and
then here's the trick we take that
markdown and we split the document based
on the header so we split by hashtag and
this way we can easily split any PDF
document up into sections where all of
the context of this section is is
relevant versus just chunking by for
example thousand characters and
splitting something up Midway so use
this to your advantage this is really a
really great tool it's cheap it's it's
fast it's it's great okay then for web
applications so whenever we have to put
something an application Live where it
can run in the background and clients
can access it or it can listen for for
web hooks triggers Etc we use fast API
so this is our framework of choice it
works on top of the penic uh schemas so
that helps because we also leverage that
for our large language model so penic
plays a core role in how we Define our
data models and how we validate our
applications with which overall
increases the robustness of your
applications all right so then next if
the application requires scale meaning
lots of users maybe even like multiple
organizations multiple users you can
Implement something like task queuing we
use salary for that if scaling is needed
so this lets you create a task cue where
every time a request comes in we use a
separate like Q to place that new
request in and then salary just handles
that step by step in order to make sure
your system does not overload quick
example if you have your application and
all of a sudden 1,000 requests come in
it's very likely that the surfer that
you're uh running that on is not going
to process that very well that's where
something like salery can come in let's
see where was I okay then large language
model monitoring we use l views having a
way to monitor your your applications in
terms of traces meaning you can
literally see the prompt and the data
that was sent to the model to the large
language model is really a must for
debugging and monitoring at scale so we
use Lang fuse because it's full and open
fully open source so we self-host it uh
it's great it's very similar to Langs
Smith okay then front end Development A
lot of these applications also require
some kind of front end for the client to
interact with not always but it it
usually it happens so either they have
to trigger a pipeline or they have to
upload some documents they want to start
a process if you want a simple UI you
can look into streamlet which is nice
because uh you can build front ends
using pure python so it's a language you
probably already know so you can get
started with that if you want more
complex UI you can develop something
custom with JavaScript react nexas uh
but this is for a lot of people probably
watching data scientist machine learning
Engineers uh this is not our default our
default stack I don't know yavas script
so I always partner up with someone if
this is required another great tip
sometimes air table is all you need air
table is really awesome it's a low code
database tool it's basically Google
Sheets in Excel on steroids it works
really great but the cool thing is you
can also create forms so sometimes that
is all you need and you can trigger web
hooks you can trigger automation so uh
you can even have custom code uh trigger
in intermediate steps so sometimes that
is all you
need databases we personally if we need
a structured database we use postgress
for unstructured databases we use
mongodb and again we just follow the
open source principle here and then for
Factor databases uh you can look into
your quadrant or for simpler use cases
uh you can look into postgress with PG
factor which is actually what we running
right now for a rack application that we
have live quadrant is a factor database
specific database so it has a little bit
it has more functionality specifically
for uh querying filtering Etc but
sometimes this is all you need and it's
really straightforward to get started
with that all right and then let's get
into project management so up until this
point I've talked a lot about Frameworks
tools techniques that we use to consider
this core project pattern input
processing output and tie everything
together this is the stack that we're
using but how do you manage these
projects which becomes more and more
important as you start to work with the
team but even if it's just you this is
also important for verion control we use
GitHub but we're on the paid plan to
protect our branches so for example we
can just straight push to the main if
that is uh the production Branch for one
of our applications you have a little
bit more uh control when it comes to
security privileges all of that so we
just use kup but you can also use what
are the other ones you have bit bit
bucket as your devops but make sure you
have a solid Version Control in place
and then when it comes to task
management you can use a simple planning
board so you can use giup issues you can
use Trello jira linear or clickup that's
what we're using and that's also the
sponsor of today's video if you're
anything like me juggling all your tasks
documents and projects can be a real
headache that's where clickup comes in a
single app to bring all of this together
now I personally love clickup because it
lets me manage everything in one place
with customized views seamless
Integrations and realtime updates also
as a developer I love the giup
integration and automations to manage my
development projects and creating and
sharing documents for my team or for my
clients has never been easier I've tried
all the other apps like Trello notion
jira or linear but to me there was
always something missing and clickup
just made sense so if you're looking to
boost your product ity whether that's
for personal projects or team
collaborations give click up a try it's
made a huge difference for me click the
link in the description to get started
so yeah clickup is awesome so even this
document that I'm sharing right now all
clickup so just I love how we can do
everything in one place inside of our
company but that concludes part two how
to build these generative AI projects
and again I did a high level overview of
the stuff that we're using this is
endlessly infinitely complex and again
each video could have a separate part on
its own so make sure if you want to know
more about a certain topic framework let
me know in the comments then let's get
into part three and this is like I've
said this is the most challenging of
them all how to deliver generative AI
projects because like I've said building
something on your machine that runs when
you click play or run script is
something entirely different from
something that you put into production
on a server where it just runs clients
can use it client can clients can access
it and most importantly it adds value to
the company on a consistent basis so
let's talk about that and let's talk
about some principles and things to keep
in mind so going from local PC to
production is hard data scientists are
not software Engineers this is a tricky
one so I have a data science background
I'm learning a ton about software
engineering so for the past year that
I've been one and a half year already
that I've really been diving deep into
this I've learned a ton but I still
wouldn't consider myself a software
engineer so whenever it comes to the
point where we really have to put things
into production this is where I partner
up with a true software engineer who
knows this stuff and who's done it
before next hallucinations become hard
to Monitor and fix at scale writing your
prompt putting in your data running a
couple of tests locally is all great but
when you have a system that has 100
maybe even thousands of of queries every
day there are going to be hallucinations
especially in the beginning and spotting
those
hallucinations is pretty
challenging and that becomes really hard
and this is also where where tools like
Lang views can come in and where you
should really closely monitor your o
system overall in order to make sure
that it remains stable another tricky
thing is okay what okay let's say you
spot a hallucination
you change the prompt now you fix the
hallucination but changing this prompt
by maybe adding this one sentence don't
do this or specifically do this or that
how is that going to affect the system
as a whole these are all questions and
challenges that everyone right now
working on these type of project is
literally figuring out how to manage
this at scale it's challenging also
remember software requires constant mon
monitoring and maintenance
many companies don't have the right
mindset for these ongoing costs and
improvements a lot of clients they come
to you they have a request we want to do
something with AI we want to build this
automation blah blah blah cool but they
expect you to just come in uh usually
they they think that it's actually way
cheaper than it actually is these
projects are expensive so you have to
first elaborate on that but then also
explain them that maintaining those
applications and improving them is also
also going to be pricey and now this is
going to be a tradeoff between are they
going to self host do they already have
the infrastructure in the Personnel in
place for you to literally come in as a
developer and then hand it over to them
then of course it becomes a lot easier
but if you have if you have to self host
becomes more tricky let's get into that
hosting options so you should decide
what what's the best fit for this
project so letting the client Host this
or self hosting we do both sometimes our
clients come to us and they say Dave we
already have an measure environment uh
we can give you access develop in our
environment and we'll maintain it we'll
take care of the costs that's on one
part that's great because then the
infrastructure is already there on the
other hand if clients come to us like
you ask them about infrastructure cloud
provider they have no idea no we don't
have that okay you can offer to self
host so self host meaning we can offer
that we host the solution so this is
more work something you should be
prepared for but but it's a chance an
opportunity for recurring Revenue so
that could be an interesting one but
then you should also really consider
this do you do you actually want to do
this because it can be stressful and you
need the right skills so this could also
be the point really where you partner up
with someone that has experience with
that so all things to consider all right
then let's get into Cloud platform so
like I've said we use Microsoft Azure uh
that is just mainly because most of the
big companies and clients that we work
with here in the Netherlands use
Microsoft Azure but you can also
consider AWS or Google cloud cloud I
know it in the US ad ad is a lot bigger
pick something that works for you in
general they all high level can do the
same stuff deployment and maintenance so
we recommend if you put things in
production to use Docker for consistent
and scalable application deployment so
take everything that you've built put it
inside a a Docker file a Docker
container and then the deploy it through
that environments if it's really a big
project affecting lots of users
potentially I would always recommend
splitting it up into a production and a
test environment so you have your own
production environment running where
clients users are interacting with and
you also have a test environment where
first you develop then you push it to
test see make sure that everything runs
stable and then you push it to
production then your cicd pipeline so we
use giup automation to streamline the
deploy deployment process so we create
sep seate branches to push to separate
servers for example so to the production
or the test environment uh it's all
automated you can set it up using giup
automations then testing so make sure to
implement unit test so to so you can
sure the reliability and functionality
of your application when you start to uh
improve add new features then also again
the cost transparency be clear about
server SL API cost and maintenance fees
so for this can sometimes be a little
bit tricky but all these Cloud providers
they have calculators that you can use
to say okay what resources do you use
what's the plan that you're using what's
the volume and you can get a ballpark
estimate of what this is going to cost
first and foremost at the server level
so just running the Surfers running the
infrastructure and then with these AI
applications where also a lot of the
costs depending on the scale of the
application come from is just the the
API cost of doing the calls to either
open AI to clot or to Azure open
and for this you ideally with a client
want to set set up a monthly payment
structure so be clear about this uh if
it's in their invironment you don't have
to deal with this maybe only just a
maintenance fee or just talk talk or
explain to them okay if if you ever need
me for maintenance improvements this is
my hourly rate and you can do it a
little bit more flexible on Dem mon or
you could offer a set hours per month or
per week that you are available and put
them on a retainer different way ways to
go about that monitoring we use Sentry
which I've personally recently
discovered this is great so Sentry is a
monitoring tool and it has connectors
templates for all kinds of Frameworks so
it also has a framework uh or template
whatever you would call that for fast
API and basically the thing that Sentry
does is uh whenever something in your
application results in an error it's
going to lock that and we have it hooked
up to our slack Channel and now when we
put this into when we have an
application running into production and
there is for example a new uh there is a
new uh example where there is some data
missing and through that a function
errors because there is a uh what what
really happens often is for example that
the key is not available so you get a
key error on some of your data uh that's
very common with these at least
applications that we're building it
shows up and then we can catch that make
it optional more of do or perform a
check Etc so we use Sentry for that
great tool okay finally security metrics
and then I want to dive into an example
architecture that we've recently worked
on for a project and that you can
hopefully take a little bit of
inspiration from but security message
also really important so first of all we
have API keys and credentials so you're
working with clients they're sharing
information with you make sure to store
and treat all of this information
carefully we use one password to just
store it internally so whenever a client
shares an API Key password whatever we
use one password then also for all the
authentication methods or connections
that you have on your application make
always make sure that there is some form
of multiactor
authentication so this is for for
databases for apis for web hooks uh so
so use a header key for example uh so
make sure that that is all safe and of
course this gets more and more important
dependent uh depending on on the
sensitivity of the data that you're
dealing with if it's really sensitive
data you can also consider uh VPN to
really put your application on Lock and
block everything except for one WID
listed IP or Azure has something where
you can create an internal VPN Gateway
and it can just connect to that so
security measures really important and
the thing is also for a lot of people
working on these
applications not their core Str
I see a lot of people either data
scientists data Engineers uh machine
learning Engineers enter into this field
and you really want to consider all of
the software development best practices
when you're going to put applications
like this into your production also for
people just in general new to the space
new to AI new to software
development these are all things that
you you have no idea about and I say
that because I still am like literally a
noob to most of these things and I've
been doing this this for 5 years already
so make sure that you understand what
you don't know and partner up with
people that do know that stuff in order
to make sure you can just provide
reliable solutions for your clients all
right and then let's get into an example
architecture diagram of a project that
we're literally working on right now and
I don't see a lot of people talking
about this and I'm just going to give it
away either way but this is I think in
my opinion such a cool such a nice
blueprint for lots of generative AI
solutions that that you can build right
now this the only thing that that's
missing is that this doesn't have a
front end that's because we're
integrating with another application
that we're uh getting data from and
sending data through but other than that
this is a really scalable robust and
secure framework that that we're working
with right now so high level it's it's
mostly just a a summary of of everything
that that I've discussed in part two in
part three of this video but for this
particular solution we're connecting
with an existing application so we're
using their API and we're using HTTP
triggers whenever something happens in
the system in that application we can
set up uh a trigger and it hits our back
end so it sends it sends a request to
our fast API backend and from there we
get all the data and that is where the
the pipeline is triggered so we use
queuing mechanism uh in here and we use
Reddit as an intermediate step I think
that's the only thing that I didn't
mention in uh in this dock in the video
in the document and through redis we
sent it to to celery to put it into a
queue we can also visualize that using
flow and then we set uh the output is
then uh put into the database so in the
postc SQL database so this is how we lck
internally all of the data that we're
using uh later for referencing for
monitoring and we enrich the data as
well but the the back end and the Q
worker that is where all the the magic
is happening in terms of the processing
so we take that data we we take it
through a data processing pipeline where
we perform various data enrichments and
also uh multiple large language model
queries in between to then finally get a
desired output and that output is sent
back into the system that we're dealing
with so this is a architecture that you
can copy for for lots of use cases where
you integrate AI into a current system
which is a really good use case now next
to that here we have a separate Resource
Group where we also have l view so we
talked about that then we also this is
the production version we also have the
test version which is a copy a literal
copy of what we have running in
production the only difference is that
it cannot send information back to the
source application and then the the cool
thing about this where when it comes to
security we put so we host all of this
in Microsoft Azure and we put it into uh
we created an Azure virtual Network and
then created the VPN Gateway and we only
allow incoming request through this VPN
Gateway and that means when we as
developers want to work with this we
with our laptops can connect with this
VPN Gateway and that is the only way to
connect with this and connect with these
resources anything else will be blocked
by default so this is how you can
entirely shut off your application so no
one can reach it because uh we're
dealing with sensitive information in
this uh in this database in this whole
system and we don't want uh that to be
interrupted of course and then here we
make sure we have the uh we have some
some type of multip multiactor
authentication so we send the request to
our back end but we also have an
authentication through a header in there
that we implement it so this is a really
cool in my opinion an architecture and
not all props to me this is really the
the software engineer that that I'm
currently working with that has
engineered this for the project that
we're working on so that concludes this
video so I hope this was helpful it's a
pretty long one I really went in depth
talking about how what we currently do
inside our company data Lumina so if you
found this video helpful please leave a
like and also consider subscribing if
you want to learn more about
uh what we're doing how we're helping
our clients or want to learn more about
freelancing how to get started with that
so make sure to subscribe if that sounds
like you and then for now make sure to
watch this video next that's the video
where I dive deeper into the client
acquisition part how you can find your
own clients how you can get started with
tools resources books that you can dive
into so make sure to check that out next
and then I'll see you in the next one
関連動画をさらに表示
5.0 / 5 (0 votes)