How to Find, Build, and Deliver GenAI Projects

Summary

This video script offers a comprehensive guide for engineers and developers on finding, building, and delivering custom generative AI solutions. It covers tactics for project discovery, the tech stack and processes for development, and the challenges of production deployment. The speaker shares insights from his company, Data Lumina, and emphasizes the importance of prompt engineering, leveraging social media for lead generation, and maintaining applications at scale.

Takeaways

• 📚 The video outlines a comprehensive process used by Data Lumina for building custom generative AI solutions, tailored for engineers and developers interested in AI applications.
• 🛠️ The process is divided into three main parts: finding generative AI projects, building these projects, and delivering them to clients.
• 💡 When starting, it's recommended to offer general AI services and then specialize (niche down) based on client needs and feedback.
• 🔎 Finding projects involves creating a clear offer, using acquisition methods like social media and networking, and effectively selling the project through understanding client needs and building trust.
• 🤖 Building generative AI projects requires understanding the core pattern of inputs, processing, and outputs, and validating ideas locally before scaling.
• 👨‍🏫 Prompt engineering is crucial and should be based on proven frameworks and templates to ensure the effectiveness of AI models.
• 🛑 Minimize AI usage by using LLM processing as a last resort, and focus on maximizing functionality through regular code before resorting to AI.
• 🔧 Utilize tools like Azure Document Intelligence for tasks like document parsing, and FastAPI for web applications to streamline development.
• 🔒 Security is paramount, especially when dealing with sensitive data, and multifactor authentication should be implemented for all connections.
• 🌐 The transition from local development to production requires careful consideration of hosting options, cloud platforms, and CI/CD pipelines for consistent deployment.
• 📈 Ongoing monitoring and maintenance are essential for generative AI projects, with tools like Sentry for error tracking and Lang fuse for monitoring LLM interactions.

Q & A

What is the main focus of the video by Data Lumina?

The video focuses on guiding engineers and developers on how to find, build, and deliver custom generative AI solutions for clients, rather than using no-code tools.

Who is the target audience for this video?

The target audience includes engineers, developers, independent contractors, and freelancers interested in building and exploring generative AI applications.

What are the three main parts the video is divided into?

The video is divided into three parts: finding generative AI projects, building these projects, and delivering them to clients.

What does Dave Abar recommend for someone just starting out with generative AI solutions?

Dave Abar recommends starting general, talking to clients to understand their needs, and possibly working for free in return for testimonials and referrals to build confidence and a portfolio.

How can one leverage social media platforms for finding clients interested in generative AI?

One can leverage social media platforms like LinkedIn and YouTube by sharing experiences, learnings, and insights related to generative AI to attract potential clients.

What is the importance of the discovery call in the sales process?

The discovery call is important for understanding the customer's pain points, offering advice, and building trust, which are crucial for converting leads into customers.

Why is prompt engineering considered as important as coding in generative AI projects?

Prompt engineering is as important as coding because the quality of prompts can significantly impact the output of the AI model, affecting the overall effectiveness of the application.

What is the significance of using a version control system like GitHub in managing generative AI projects?

Using a version control system like GitHub helps in managing changes, protecting branches, ensuring security, and facilitating collaboration among team members.

What are some challenges faced when moving generative AI projects from a local machine to a production environment?

Challenges include ensuring software reliability, dealing with hallucinations in AI outputs at scale, constant monitoring and maintenance, and managing ongoing costs.

How does Data Lumina handle the integration of AI into existing systems in their projects?

Data Lumina uses an architecture that involves connecting with existing applications via APIs, using queuing mechanisms, data processing pipelines, and integrating outputs back into the system.

What is the role of task queuing in scaling generative AI applications?

Task queuing, implemented using tools like Celery, helps manage the load by creating a queue for incoming requests, ensuring the system does not get overloaded and maintains performance even under high demand.

What are some best practices for monitoring and maintaining generative AI applications in production?

Best practices include using monitoring tools like Sentry, setting up CI/CD pipelines for automated testing and deployment, implementing unit tests, and being transparent about costs associated with server and API usage.

Outlines

00:00

🛠️ Building Custom Generative AI Solutions

Dave Abar, founder of Data Lumina, introduces a video guide for engineers and developers to build and deliver custom generative AI projects. The video is divided into three parts: finding projects, building solutions, and delivering them. Dave emphasizes that the video is not about using no-code tools but about creating tailored AI applications. He shares tactics for finding projects, building with specific tech stacks, and the challenges of production deployment. The audience is encouraged to follow along with a document provided in the description.

05:02

🔍 Finding Generative AI Projects and Building Confidence

The second paragraph focuses on the initial steps for finding generative AI projects, which include defining the offer, exploring acquisition methods, and understanding sales strategies. Dave suggests starting with a general approach, gaining experience, and researching successful use cases. He also recommends offering services for free in return for testimonials and referrals to build a portfolio and confidence in delivering value with generative AI.

10:03

📞 Acquisition and Sales Strategies for AI Projects

Dave discusses tactics for acquiring leads and converting them into clients. He advises leveraging current networks, social media platforms like LinkedIn and YouTube, and Discord communities. Dave also highlights the importance of providing value over direct sales pitches on social media. For sales, he emphasizes the difficulty of converting leads and suggests having strategies and experience in place. The process involves discovery calls, creating proposals, and discussing contract terms and payment schedules.

15:04

🚀 Leveraging Freelancing in the Age of AI

In this part, Dave explores the impact of AI on freelancing, citing statistics that show an increase in productivity and opportunities for tech professionals to build solutions for startups and enterprises. He mentions that generative AI makes freelancers more likely to pursue independent careers, and he shares his personal experience transitioning from freelancing to running a full development company due to the demand for generative AI solutions.

20:06

💡 Fundamentals of Building Generative AI Projects

Dave outlines the core pattern of generative AI projects, which involves inputs, processing, and outputs. He stresses the importance of validating ideas locally, creating flow diagrams, and taking prompt engineering seriously. The paragraph also covers the significance of data quality and the use of large language models, with a preference for Azure Open AI due to data safety assurances.

25:06

🛑 Minimizing AI Usage and Utilizing Tools for AI Projects

The paragraph discusses best practices for integrating large language models into applications, advocating for the use of AI as a last resort after all other coding solutions have been exhausted. Dave recommends using Pinecone and Langchain for robust integration and emphasizes the importance of monitoring applications at scale using tools like Langfuse. He also suggests leveraging Azure Document Intelligence for parsing documents and outlines the tech stack used for building generative AI solutions.

30:09

🤖 Project Management and Tools for AI Development

Dave covers project management tools and techniques for developing AI projects, including version control with GitHub, task management with platforms like ClickUp, and the importance of automation. He highlights ClickUp's features and integrations for task management and emphasizes the need for a solid project management approach to handle complexity and ensure project success.

35:11

🌐 Delivering Generative AI Projects: Challenges and Solutions

The paragraph delves into the complexities of moving AI projects from local development to production environments. Dave talks about the challenges data scientists face in software engineering, the importance of monitoring for hallucinations in AI outputs, and the ongoing costs of maintenance. He also discusses hosting options, the use of Docker for deployment, and the need for testing, monitoring, and security measures.

40:11

🔒 Security and Example Architecture for AI Projects

In this section, Dave discusses security measures for AI projects, including the use of multifactor authentication and secure storage of API keys. He presents an example architecture diagram for a project currently in development, which includes a secure and scalable framework using Azure services, VPN for access control, and a focus on security at every layer of the application.

45:14

🏢 Conclusion and Next Steps for Generative AI Projects

The final paragraph concludes the video with a recap of the in-depth discussion on building and delivering generative AI projects within Data Lumina. Dave invites viewers to like and subscribe for more content on freelancing and working with clients. He also encourages viewers to watch the next video for further insights into client acquisition strategies.

Keywords

💡Generative AI

Generative AI refers to artificial intelligence systems that are capable of creating new content, such as text, images, or audio, based on input data or prompts. In the video, this concept is central to the discussion around building custom AI solutions for various business applications, emphasizing the creative and innovative potential of these systems.

💡Custom Solutions

Custom solutions are tailor-made products or services designed to meet the specific needs of a client or a business. The video focuses on the process of developing such solutions in the realm of generative AI, highlighting the importance of understanding client requirements and creating unique applications to address those needs.

💡Data Lumina

Data Lumina is the company mentioned in the script that specializes in building custom generative AI solutions for clients. The video discusses the company's approach and methodologies in finding, building, and delivering AI projects, positioning Data Lumina as an expert in the field.

💡Acquisition Methods

Acquisition methods in the context of the video refer to the strategies used to find and secure leads for AI projects. The script outlines various tactics such as leveraging social media, engaging in Discord communities, and participating in professional networks to attract potential clients.

💡Sales

Sales, as discussed in the video, involves the process of converting leads into customers by effectively communicating the value of a generative AI project and building trust. The script emphasizes the importance of understanding the customer's needs, offering advice, and providing a clear proposal with defined scope and pricing.

💡Prompt Engineering

Prompt engineering is the process of carefully designing the prompts or inputs given to generative AI models to elicit desired outputs. The video underscores the significance of this process, suggesting that a well-crafted prompt can be as crucial as the code itself in determining the success of an AI application.

💡Azure Open AI

Azure Open AI is a service mentioned in the script that provides access to AI models and capabilities through the Azure platform. The video discusses its use in building generative AI solutions, noting the benefits of using a trusted provider that ensures data safety and compliance.

💡APIs

APIs, or Application Programming Interfaces, are sets of rules and protocols that allow different software applications to communicate with each other. In the video, APIs are discussed in the context of integrating AI models into applications and the potential costs associated with using third-party services.

💡SaaS (Software as a Service)

Though not explicitly mentioned in the script, the concept of SaaS is implied in the discussion of delivering AI solutions. SaaS refers to a software delivery model in which software is provided over the internet, on a subscription basis. The video touches on the idea of offering AI applications as a service to clients, which aligns with the SaaS model.

💡Freelancing

Freelancing is a work arrangement where a person offers services, such as AI project development, to clients on a flexible and independent basis. The video discusses the opportunities for developers and engineers to explore generative AI applications as freelancers, highlighting the potential for building a business in this niche.

💡Project Management

Project management involves the organization and coordination of activities in a project. The video mentions tools and practices for managing the development of generative AI projects, such as using GitHub for version control and ClickUp for task management, emphasizing the importance of structured project management in delivering successful outcomes.

Highlights

Dave Abar introduces a comprehensive process for building custom generative AI solutions at Data Lumina.

The video is tailored for engineers and developers interested in exploring AI applications, not for those seeking guidance on no-code tools.

A three-part structure covers finding projects, building AI solutions, and delivering them effectively.

Dave emphasizes starting generative AI projects with a general approach, understanding client needs, and then niche-ing down.

The importance of having a clear 'offer' when starting out in generative AI projects is discussed, including choosing between a general or niche approach.

Tactics for finding leads include leveraging social media, engaging in Discord communities, and participating in educational groups like Data Alchemy.

Sales strategy involves understanding the customer's pain points and building trust through expertise and valuable insights.

The video discusses the importance of prompt engineering in generative AI and the significance of data quality over the AI model used.

Dave recommends using Python for backend development, Azure Open AI for model provider, and emphasizes minimizing AI usage until necessary.

An example architecture diagram is presented for a robust and secure generative AI project integration.

The video covers the complexities of moving from a local development environment to a production environment for AI projects.

Different hosting options are explored, including client-hosting and self-hosting solutions with considerations for recurring revenue.

The use of Docker for consistent and scalable application deployment is recommended, along with setting up CI/CD pipelines.

Security best practices such as using multifactor authentication and VPN connections are highlighted for secure application development.

The video concludes with a call to action to subscribe for more insights on freelancing and AI project management.

Transcripts

how to find build and deliver generative

AI projects in this video I'm going to

walk you through the entire process that

we use inside of our company data Lumina

to build custom generative AI solutions

for our clients now on that note this

video is not for you if you're looking

to learn more about how to build and

deliver generative AI Solutions using

nood tools like sapper make voice flow

or bpress which are all great by the way

but this video is really going to dive

into building custom Solutions so this

video is for engineers and developers

who want to build and explore generative

AI applications in their own company or

for example do that as an independent

contractor or freelancer to really start

your own thing which is really a great

opportunity right now for every

developer that is really interested in

working with generative AI this video is

split up into three parts first we'll

look at how to find generative AI

projects we'll share some tactics that

we use inside of our company then how to

build where really get into the

specifics the Tex stack the the

Frameworks the processes the tools that

we use everything and then we talk about

how to deliver these generative AI

projects because it's one thing to build

something locally on your laptop that

works but bringing them putting that

into production where it can run and the

client can access it that's a whole

different story and that's the most

challenging part so that's how it is

split up quick context about me for

those of you that don't know me my name

is Dave abar I'm the founder of data

Lumina and we help organizations and

individuals to navigate the AI

Revolution and we do that by building

custom Data Solutions for the clients

that we work with and we also cultivate

technological leadership through our

educational resources that is our

YouTube channel our free community data

Alchemy which you can check out and our

paid Community data freelancer where we

help data professionals to get started

with freelancing by showing them how we

run our company how we find clients and

collaborating with them now if you want

to follow along with this document you

can click the link in the description

you'll uh see a link to this document if

you click it you can duplicate or

download it after signing up for clickup

so if that's something you want you can

do that you can also export it to

markdown to put into notion obsidian

whatever tool that you are using so with

that out of the way let's get straight

into things and start with part one how

to find generative AI projects I have a

diagram over here and a stepbystep

instructions uh let's first break it

down really at the core most top

fundamental level what do you need in

order to find these projects it's split

up into three parts you have the offer

you have acquisition methods so how to

find leads and you have sales after

finding those leads how do you actually

sell the project to them so let's start

with the offer this is really what are

you going to sell and when you're

starting out you should ask yourself the

question are you going to go general or

are you going to Niche down General

really is okay what's your problem you

talk to the to the clients they say hey

we have problem X Y and Z and you say

hey I can build that for you going Niche

is you you go to the client say you have

a problem I have the solution you go

specific you know a certain problem that

for example a niche or a market is

dealing with and you built a tailored

solution for that if you're just

starting out and that's probably you if

you're watching this video then I would

recommend start general first just talk

to people and let them tell you what it

is that they want and then figure out if

you can help them so how do you start

and for this you need to be experienced

so you need to be a developer a data

scientist machine learning engineer

software engineer data engineer and

ideally have some experience with

generative AI under your belth because

building custom Solutions is hard and

you you can start by for example solving

a problem that you are experienced this

is a great way to start something else

that you can do is research successful

gen fi use cases and I've listed some of

them here where you could really think

about the main departments that every

almost every company has so Marketing

sales support and operations these are

really for all of these departments

generative AI can be great and there are

lots of solutions so take some

inspiration from there see if there's

something that you like that could also

be a great starting point so you can

either just pick something on your own

start with that to build your own

portfolio or what you could also do very

powerful offer to work for free in

return for testimonial and some

referrals so go out to some people in

your Network a company your boss uh

clients you've worked with in the past

doesn't matter find a company that is

interested in doing something with

generative Ai and see if you can help

them out maybe do it for small fee maybe

do it for free but really the important

the the goal here really is to get that

case study and ideally a testimonial to

first and foremost also build confidence

so you know hey I can do this I can

actually deliver value to companies

using the skills and uh that I have and

these tools that is the offer what are

you going to sell get clear on that I

would recommend starting General and

then from there we get into the next

step which is acquisition so how to find

and convert leads and now if you're here

because you just want to know how to

build these projects or you want to do

something within your company then you

can kind of like skip over these steps

and skip this video a little bit uh

where we get into part two but if you're

new to this and you're looking to find

clients this is where you should start

so with acquisition here are some tips

to get started with this and I will

leave some I will link some more

resources later but high level start

with your current Network this is the

most uh this is the easiest way to start

so especially if you already have a case

study you have some experience just

reach out to people in your current

Network either via email direct messages

WhatsApp LinkedIn DMS and ask if the

solutions you've built could be

interesting for them or someone they

know the cool thing about generative AI

right now is that if implemented

correctly of course it can really add

value to any company any industry it

doesn't matter every company has

Marketing sales operations support and

gener that's really where generative AI

shines and then it's really about

getting the right use case making it

small and tangible enough to solve a

specific problem so start with your

network next to that I would recommend

leveraging social media platforms

LinkedIn YouTube are the two best uh

ways to get started I understand YouTube

is not for everyone can be a big step

but it's highly effective LinkedIn a

little bit easier to get started with

you just write a post a lot of people

then ask okay but what do I share what

do I write about the best thing that you

can do is share your experience and

share the things that you've learned

because in doing so you let people know

that you're actively working on this

you're learning uh you're learning about

this and people over time will naturally

start to reach out to you asking like

hey I'm actually interested in a you

talk about ax all the time can you help

me with this that's how it works it's

really powerful and even even like one

LinkedIn post could get the ball rolling

so don't underestimate this then another

important principle to keep in mind when

posting on social media don't sell

directly give value so don't go out on

LinkedIn for example asking all the time

like hey I'm looking for clients to work

with can you help me now occasionally

you can do this but the long-term play

is to give value so share what you've

learned share interesting insights share

knowledge share news and over time

people will naturally start to reach out

to you that's the principle that you

should keep in mind another way to find

leads is by by actually posting in

Discord communities for popular

Frameworks and tools that you're for

example working with so this is a very

interesting one and is gaining more and

more popularity but by just showing up

there and answering the technical

questions people might have they will

naturally again start to reach out to

you asking like hey I'm actually working

on this project I have this question

about this particular framework here I

give an example of Fai and then they

they come into that Community to look

for answers and if you see that they're

the expert and they're willing to pay

you some for that they might hire you as

a consultant or a developer to start

working with it interesting place for

developers to find leads then another

great way to connect with people and

share insights is via school so you can

for example come to data Alchemy which

is my free group I will leave the link

in the description uh we have over

15,000 members over here and it's

actually quite funny the people that

always show up here in the community and

that post a lot that add value people

will naturally start to reach out to

them and ask for help so for example

that happened to to Brandon uh multiple

times already because he's really active

in here and that could be a great way to

get your foot in the door as well and

then finally you could look into

freelance platforms like Fifer or upwork

but please note while these platforms

are great they can work in the beginning

it's really challenging to get started

with that all right and then let's get

into the next part which is sales so how

to sell a project and the thing that I

want you to keep in mind here is finding

leads is easy sales is hard if you don't

have the right experience or strategies

and the interesting thing here is that

everyone thinks that finding potential

customers finding leads is hard because

this is the number one question that uh

people ask when for example they come to

us on a discovery call for data

freelancer number one question that they

have is how do I find clients I always

tell them finding leads finding clients

is very easy you can literally walk up

to any business owner right now probably

and ask them hey is AI on your road map

do you want to do something with AI

probably all of them almost all of them

will say yes we're not sure but we know

AI is there we want to do something with

this that's a lead they're interested

but now converting that lead into a

customer I.E sales by now coming in

stepping in saying like here's how I can

help you here's the price here's what I

can do building confidence building

trust that is a lot harder again if you

don't have the right experience and the

right strategies so how do you do it

correctly and of course I'm going to

give you a summary here all of the

topics that we're discussing here could

have a separate 60-minute video on its

own but this video the goal is to give

you an outline if at any point in this

video by the way you have questions or

you want me to dive deeper into certain

topics leave a comment I'll consider it

for a future video or I I can I can

answer you in the comments but here's

how we do it at data Lumina currently

and for this I'm going to scroll up to

this image over here let me see yeah so

we talked about the offer we talked

about the acquisition strategies and and

by the way for our company right now

just in case you're curious all of our

clients come through either LinkedIn

YouTube or referrals or previous clients

that's where we are currently at so we

don't have to do any called Outreach

send messages that's how powerful social

media can be but now let's get into

sales so sales for us at least for most

companies starts with the discovery call

so this is an online call we do that via

Zoom we let people schedule via kly um

we also sometimes let them fill out some

questionnaires so

uh what's your company about what's the

problem what's the budget so we get some

data points already and on the call you

shouldn't really think about it as a

sales call because really while it is a

discovery call really the core thing

that you should do is ask questions lots

lots of questions so you don't open open

up the call and say hey I'm an I'm a

developer I have this I can do that I

can do that I can do that do you want to

buy from me no that's the last thing you

want to do a successful Discovery call

is asking as many questions as possible

to first of all really understand the

pain points of the customer understand

that you can actually help them

understand that it's the right fit and

in doing so you also build trust because

you show that you're an expert you ask

the right questions you trigger them

next to that you can also offer advice

the funny thing is a lot of

companies they have no idea what they

want the thing that they they come up

they they show up to you on a Q on a

discovery call and they say we want to

do things with AI sometimes it's a

little bit more specific we want to do

AI for marketing we want to do AI for

Content but they they have no idea so

you should then also step in as the

expert and offer advice this again

builds trust higher chance that the sale

will come through it's also important on

that initial call to talk about some

price expectation hey are you sure you

want to do this the scope that we're

talking about right now this could very

well go into the 15 15 20K depending of

course on the size of the project make

sure that they are aware of the the kind

of range that that J you're at for such

a project now later you can see the

arrow over here we after the call you

can create a specific proposal because

because a lot of the times you get a lot

of data on that call by asking all of

those questions and then you need to

take take a step back and dive in and

really create a thorough proposal so

this is where you really where you

outline scope timelines prices

deliverables and then you put a price on

it this can either be a fixed price so

you say okay this is a 10K project it's

going to take me four weeks this is what

I'm going to deliver you can do

sprint-based pricing where you say okay

we do two week Sprints uh price for

Sprint is X or you could say here's my

houry rate I think it's going to take me

around 40 to 50 hours maybe and we'll do

it like that there are multiple ways to

go about this and you should figure out

something that works for you what we

usually like to do is fix fix prices in

set intervals so for example we sell a

proof of concept we sell an MVP we for

example put four weeks on it it 6 weeks

on it Etc to clearly Define the scope

but talk about it on the first call so

they know what to expect also important

talk about potential hidden costs API

cost management cost hey do you know

actually if we if we implement this

solution for you do you know that we can

build this there's an upfront cost for

the development but also there's going

to be maintenance cost API costs give ex

set set the ranges for them give the

expectations so they know oh it's not

only upfront I pay you one time but this

is an ongoing thing because maintaining

and improving custom Solutions like this

a lot of work it's challenging it takes

time so you need to get paid for that as

well usually what we like to do once we

create the proposal which we usually do

either in a simple document or

PowerPoint presentation we discuss the

proposal on a second call or share it

via email we found that uh discussing it

on a second call where the client is

there works better because if you share

it via email they might have lot lot of

questions um they're not sure about

things and sometimes they just ghost you

and then next step if everything is all

right you can sign a contract so put a

contract in place uh we use doku sign

for that simple contract so let them

sign the contract and also consider a

payment schedule what you can do is you

can do for example 50/50 split 50% up

front 50% afterwards by weekly payments

uh payment uh after successful delivery

something you can play with so that's

how sales Works high level how we do it

what you need to consider all right

right and before we get into part two

there's one more thing that I want to

show you and that's this interesting

article over here thanks to AI we're in

the Golden Age of freelancing there are

a lot of interesting statistics in here

and what they found they interviewed a

bunch of top Tech professionals and they

indicated that they are all leveraging

AI first of all to increase their

productivity well this is I think what

we all should be doing but they also

capitalize on this generative AI Hye by

Building Solutions for startup and

Enterprises so they took a group of top

Tech professionals and it they it

basically said that half of them had

already built a solution a gen solution

for startups of uh or or an Enterprise

and 68% of the respondents in the survey

said that generative AI makes them more

likely to pursue an independent career

working for multiple companies it's a

really interesting article a really

interesting take out how generative a

automation reduces the need for

Freelancers in some Fields copywriters

Marketing the stuff that can really be

automated but on the other hand there

because companies notice that they can

automate this they now need Freelancers

contractors who can help them to create

these automations and this is an ideal

opportunity for for Freelancers because

these are companies that don't have ai

Engineers or data scientists inside of

their company on payroll that wouldn't

make sense they just need someone a

technical partner freelancer contractor

whatever that can help them every now

now and then to build these automations

and then they can over time replace the

overhead that they're spending on

customer support marketing Etc and now

funnel that money into potentially you

who steps in as a contractor really

interesting take and it's definitely

something that I've noticed um I've been

freelancing for 5 years already

basically whenever when uh the open AI

API came out I almost completely

switched to building generative AI

Solutions just because of how much

demand and opportun unities there is so

for my freelancing business which um is

now not even a freelancing business

anymore data Lum is now a full full

development company but I've transition

I've been able to make that transition

because of generative AI do you want to

dive deeper into how to find J fi

projects so how to find clients then you

can check out this YouTube video some

link that as well so this is how to find

freelance data and AI project in 2024

it's a 23 minute video going deeper uh

into the more specific some books some

strategies about everything that we've

discussed here all right with that said

we conclud this part one let's now get

into how to build generative AI projects

and let's start with fundamentals to

keep in mind at the core every

generative AI project follows this

pattern you have inputs you have

processing and you have outputs

generative AI generative meaning output

these models gener at stuff and they

generate stuff based on input this is

your data and your prompts and the

processing happens inside the

application so how do you bring all of

this together and there is an output

this can be in the form of text U this

can be in the form of even images this

can be in the form of any type of

combination documents you can put it

together and then also you have just the

file outputs but you also have the

destination same for the input you have

the data but you also have to Source

where it's coming from and usually what

you if you consider a full generative AI

project it combines the elements of

getting some information from a system

or an input or a form there's some type

of connection you take you take that

data you process it you add the prompts

you add the large language model you

generate something new put it into

desired format and then also put it in

uh the desired output place whether

that's on the website in a Content

management system in a on a storage

account that is generative AI projects

101 a typical flow that the applications

follow when you start to build these

projects you always want to start by

validating the ID locally meaning

getting a proof of concept you can even

sell this as a separate phase which is

what we sometimes do but you should also

be careful with that and really

communicate with the client that if you

do a proof of concept it's literally

only to test and validate f validate the

idea not something tangible that they

can use this is a mistake that that

we've made in the past where okay let's

do proof concept we're going to prove

that this works because these projects

are very Innovative Innovative they're

new a lot of these use cases are not

proven yet and sometimes there's a

little bit of ambiguity or uncertainty

whether this will actually work uh

mostly in terms of is the the quality of

the output good enough can we control

for hallucinations all of those

questions

but you want to start locally and you

should really get clear on that pattern

so get clear on the input data the

processing step the desired output in

the location and usually it helps to

create a flow diagram so create a flow

diagram you could use something like

draw.io or figma and write out the

applications flow think about where do

we need to get data from what do we need

to combine what kind of prompts do we

maybe need and then what is that output

uh going to look like then also take

prompt engineering seriously so don't

just write a prompt simply by the first

thing that comes up uh when you're

working on this or by asking C GPT to

write the prompt for you don't be lazy

here your prompt is almost as important

maybe it is as important as the code

that you're writing so follow proven uh

prompt Frameworks and templates more on

that uh in a little bit but take prompt

engineering seriously and also consider

the data is more important than the AI

that you're using with when it comes to

generative AI everyone has the same

models almost of course there's fine

tuning all of that but generally

everyone is using the same intelligence

here the difference is the data and the

data is in terms of your your literally

your input data your company data

proprietary data that that you're using

but also your prompts so combining those

two is really what makes your

application unique and potentially

better than the competition for example

all right then let's get into tools and

techniques so we use use pure python to

build our generative AI solution so we

use Python backends for prompt

engineering something that you could

look into is we like to structure our

prompts by first always focusing on a

role then a task description step by

step using Chain of Thought prompting uh

instructions we give it context we add

emotional impact and we also add some

some one uh some few shot examples and

then we tie all of this together in a

markdown format so these are proven

prompting techniques based on literature

so so if you're interested in that I

would recommend looking this up then for

large language model provider so the

brain the intelligence of your

generative AI applications uh you can

consider open AI Azure open AI or CLA we

almost always use Azure open AI because

we have the added check mark where

Microsoft ensures that they're not

sharing your data with open AI even

though Microsoft is partnered partly

owns open AI but for the clients that we

work with we can say hey your data is as

safe as the emails in your outlook inbox

um most of our the clients that we work

with they they are already using

Microsoft it's a trusted partner so that

just helps that's why we use Azure open

AI here's a big one huge one which I

also have a video on utilize penic and

instructor for robust llm integration I

will not be diving into what that is all

about but I will link this video this is

going to completely change the way you

build llm application development best

practices when it comes to large

language models and AI be Innovative but

minimize AI usage use llm processing as

a last resort what do I mean by this if

we consider the typical flow of input

processing output Etc there are

naturally going to be steps within your

application where some of these

processing steps you can just do through

a regular function maybe through regx

maybe through a simple IFL statement

make sure to

maximize that functionality first and

using an llm should really be your last

resort so really when you cannot

continue in your data processing

pipeline without AI that is where you

introduce Ai and why do you want to do

this because first of all it slows down

your application it introduce cost it

can introduce hallucination the more API

calls you have to make the more

challenging it's going to be to scale

and optimize and maintain your

application now we're talking about

generative AI application so at some

point you're going to have to ju

introduce an llm or AI because otherwise

it's not the generative AI application

but just keep that in mind because these

models are so good they can if if you

really break it down to like if you

really break problems down to the most

fundamental small part an llm can solve

almost any problem especially if you

utilize penic and instructor you can let

it analyze all kinds of problems and

then give for example a Boolean value

true false and use that to decide what

the next flow of the application should

be but really think about it can I do

this just with code first if not only

then resort to llms here's another just

random tip uh we leverage Azure document

intelligence a lot for parsing documents

so uh rack is a common use case for

generative AI

applications a problem with rack is

splitting up the data into relevant

chunks that you can then put into your

vector database for the retrieval we use

Azure document intelligence to split up

documents and let it output markdown and

then here's the trick we take that

markdown and we split the document based

on the header so we split by hashtag and

this way we can easily split any PDF

document up into sections where all of

the context of this section is is

relevant versus just chunking by for

example thousand characters and

splitting something up Midway so use

this to your advantage this is really a

really great tool it's cheap it's it's

fast it's it's great okay then for web

applications so whenever we have to put

something an application Live where it

can run in the background and clients

can access it or it can listen for for

web hooks triggers Etc we use fast API

so this is our framework of choice it

works on top of the penic uh schemas so

that helps because we also leverage that

for our large language model so penic

plays a core role in how we Define our

data models and how we validate our

applications with which overall

increases the robustness of your

applications all right so then next if

the application requires scale meaning

lots of users maybe even like multiple

organizations multiple users you can

Implement something like task queuing we

use salary for that if scaling is needed

so this lets you create a task cue where

every time a request comes in we use a

separate like Q to place that new

request in and then salary just handles

that step by step in order to make sure

your system does not overload quick

example if you have your application and

all of a sudden 1,000 requests come in

it's very likely that the surfer that

you're uh running that on is not going

to process that very well that's where

something like salery can come in let's

see where was I okay then large language

model monitoring we use l views having a

way to monitor your your applications in

terms of traces meaning you can

literally see the prompt and the data

that was sent to the model to the large

language model is really a must for

debugging and monitoring at scale so we

use Lang fuse because it's full and open

fully open source so we self-host it uh

it's great it's very similar to Langs

Smith okay then front end Development A

lot of these applications also require

some kind of front end for the client to

interact with not always but it it

usually it happens so either they have

to trigger a pipeline or they have to

upload some documents they want to start

a process if you want a simple UI you

can look into streamlet which is nice

because uh you can build front ends

using pure python so it's a language you

probably already know so you can get

started with that if you want more

complex UI you can develop something

custom with JavaScript react nexas uh

but this is for a lot of people probably

watching data scientist machine learning

Engineers uh this is not our default our

default stack I don't know yavas script

so I always partner up with someone if

this is required another great tip

sometimes air table is all you need air

table is really awesome it's a low code

database tool it's basically Google

Sheets in Excel on steroids it works

really great but the cool thing is you

can also create forms so sometimes that

is all you need and you can trigger web

hooks you can trigger automation so uh

you can even have custom code uh trigger

in intermediate steps so sometimes that

is all you

need databases we personally if we need

a structured database we use postgress

for unstructured databases we use

mongodb and again we just follow the

open source principle here and then for

Factor databases uh you can look into

your quadrant or for simpler use cases

uh you can look into postgress with PG

factor which is actually what we running

right now for a rack application that we

have live quadrant is a factor database

specific database so it has a little bit

it has more functionality specifically

for uh querying filtering Etc but

sometimes this is all you need and it's

really straightforward to get started

with that all right and then let's get

into project management so up until this

point I've talked a lot about Frameworks

tools techniques that we use to consider

this core project pattern input

processing output and tie everything

together this is the stack that we're

using but how do you manage these

projects which becomes more and more

important as you start to work with the

team but even if it's just you this is

also important for verion control we use

GitHub but we're on the paid plan to

protect our branches so for example we

can just straight push to the main if

that is uh the production Branch for one

of our applications you have a little

bit more uh control when it comes to

security privileges all of that so we

just use kup but you can also use what

are the other ones you have bit bit

bucket as your devops but make sure you

have a solid Version Control in place

and then when it comes to task

management you can use a simple planning

board so you can use giup issues you can

use Trello jira linear or clickup that's

what we're using and that's also the

sponsor of today's video if you're

anything like me juggling all your tasks

documents and projects can be a real

headache that's where clickup comes in a

single app to bring all of this together

now I personally love clickup because it

lets me manage everything in one place

with customized views seamless

Integrations and realtime updates also

as a developer I love the giup

integration and automations to manage my

development projects and creating and

sharing documents for my team or for my

clients has never been easier I've tried

all the other apps like Trello notion

jira or linear but to me there was

always something missing and clickup

just made sense so if you're looking to

boost your product ity whether that's

for personal projects or team

collaborations give click up a try it's

made a huge difference for me click the

link in the description to get started

so yeah clickup is awesome so even this

document that I'm sharing right now all

clickup so just I love how we can do

everything in one place inside of our

company but that concludes part two how

to build these generative AI projects

and again I did a high level overview of

the stuff that we're using this is

endlessly infinitely complex and again

each video could have a separate part on

its own so make sure if you want to know

more about a certain topic framework let

me know in the comments then let's get

into part three and this is like I've

said this is the most challenging of

them all how to deliver generative AI

projects because like I've said building

something on your machine that runs when

you click play or run script is

something entirely different from

something that you put into production

on a server where it just runs clients

can use it client can clients can access

it and most importantly it adds value to

the company on a consistent basis so

let's talk about that and let's talk

about some principles and things to keep

in mind so going from local PC to

production is hard data scientists are

not software Engineers this is a tricky

one so I have a data science background

I'm learning a ton about software

engineering so for the past year that

I've been one and a half year already

that I've really been diving deep into

this I've learned a ton but I still

wouldn't consider myself a software

engineer so whenever it comes to the

point where we really have to put things

into production this is where I partner

up with a true software engineer who

knows this stuff and who's done it

before next hallucinations become hard

to Monitor and fix at scale writing your

prompt putting in your data running a

couple of tests locally is all great but

when you have a system that has 100

maybe even thousands of of queries every

day there are going to be hallucinations

especially in the beginning and spotting

those

hallucinations is pretty

challenging and that becomes really hard

and this is also where where tools like

Lang views can come in and where you

should really closely monitor your o

system overall in order to make sure

that it remains stable another tricky

thing is okay what okay let's say you

spot a hallucination

you change the prompt now you fix the

hallucination but changing this prompt

by maybe adding this one sentence don't

do this or specifically do this or that

how is that going to affect the system

as a whole these are all questions and

challenges that everyone right now

working on these type of project is

literally figuring out how to manage

this at scale it's challenging also

remember software requires constant mon

monitoring and maintenance

many companies don't have the right

mindset for these ongoing costs and

improvements a lot of clients they come

to you they have a request we want to do

something with AI we want to build this

automation blah blah blah cool but they

expect you to just come in uh usually

they they think that it's actually way

cheaper than it actually is these

projects are expensive so you have to

first elaborate on that but then also

explain them that maintaining those

applications and improving them is also

also going to be pricey and now this is

going to be a tradeoff between are they

going to self host do they already have

the infrastructure in the Personnel in

place for you to literally come in as a

developer and then hand it over to them

then of course it becomes a lot easier

but if you have if you have to self host

becomes more tricky let's get into that

hosting options so you should decide

what what's the best fit for this

project so letting the client Host this

or self hosting we do both sometimes our

clients come to us and they say Dave we

already have an measure environment uh

we can give you access develop in our

environment and we'll maintain it we'll

take care of the costs that's on one

part that's great because then the

infrastructure is already there on the

other hand if clients come to us like

you ask them about infrastructure cloud

provider they have no idea no we don't

have that okay you can offer to self

host so self host meaning we can offer

that we host the solution so this is

more work something you should be

prepared for but but it's a chance an

opportunity for recurring Revenue so

that could be an interesting one but

then you should also really consider

this do you do you actually want to do

this because it can be stressful and you

need the right skills so this could also

be the point really where you partner up

with someone that has experience with

that so all things to consider all right

then let's get into Cloud platform so

like I've said we use Microsoft Azure uh

that is just mainly because most of the

big companies and clients that we work

with here in the Netherlands use

Microsoft Azure but you can also

consider AWS or Google cloud cloud I

know it in the US ad ad is a lot bigger

pick something that works for you in

general they all high level can do the

same stuff deployment and maintenance so

we recommend if you put things in

production to use Docker for consistent

and scalable application deployment so

take everything that you've built put it

inside a a Docker file a Docker

container and then the deploy it through

that environments if it's really a big

project affecting lots of users

potentially I would always recommend

splitting it up into a production and a

test environment so you have your own

production environment running where

clients users are interacting with and

you also have a test environment where

first you develop then you push it to

test see make sure that everything runs

stable and then you push it to

production then your cicd pipeline so we

use giup automation to streamline the

deploy deployment process so we create

sep seate branches to push to separate

servers for example so to the production

or the test environment uh it's all

automated you can set it up using giup

automations then testing so make sure to

implement unit test so to so you can

sure the reliability and functionality

of your application when you start to uh

improve add new features then also again

the cost transparency be clear about

server SL API cost and maintenance fees

so for this can sometimes be a little

bit tricky but all these Cloud providers

they have calculators that you can use

to say okay what resources do you use

what's the plan that you're using what's

the volume and you can get a ballpark

estimate of what this is going to cost

first and foremost at the server level

so just running the Surfers running the

infrastructure and then with these AI

applications where also a lot of the

costs depending on the scale of the

application come from is just the the

API cost of doing the calls to either

open AI to clot or to Azure open

and for this you ideally with a client

want to set set up a monthly payment

structure so be clear about this uh if

it's in their invironment you don't have

to deal with this maybe only just a

maintenance fee or just talk talk or

explain to them okay if if you ever need

me for maintenance improvements this is

my hourly rate and you can do it a

little bit more flexible on Dem mon or

you could offer a set hours per month or

per week that you are available and put

them on a retainer different way ways to

go about that monitoring we use Sentry

which I've personally recently

discovered this is great so Sentry is a

monitoring tool and it has connectors

templates for all kinds of Frameworks so

it also has a framework uh or template

whatever you would call that for fast

API and basically the thing that Sentry

does is uh whenever something in your

application results in an error it's

going to lock that and we have it hooked

up to our slack Channel and now when we

put this into when we have an

application running into production and

there is for example a new uh there is a

new uh example where there is some data

missing and through that a function

errors because there is a uh what what

really happens often is for example that

the key is not available so you get a

key error on some of your data uh that's

very common with these at least

applications that we're building it

shows up and then we can catch that make

it optional more of do or perform a

check Etc so we use Sentry for that

great tool okay finally security metrics

and then I want to dive into an example

architecture that we've recently worked

on for a project and that you can

hopefully take a little bit of

inspiration from but security message

also really important so first of all we

have API keys and credentials so you're

working with clients they're sharing

information with you make sure to store

and treat all of this information

carefully we use one password to just

store it internally so whenever a client

shares an API Key password whatever we

use one password then also for all the

authentication methods or connections

that you have on your application make

always make sure that there is some form

of multiactor

authentication so this is for for

databases for apis for web hooks uh so

so use a header key for example uh so

make sure that that is all safe and of

course this gets more and more important

dependent uh depending on on the

sensitivity of the data that you're

dealing with if it's really sensitive

data you can also consider uh VPN to

really put your application on Lock and

block everything except for one WID

listed IP or Azure has something where

you can create an internal VPN Gateway

and it can just connect to that so

security measures really important and

the thing is also for a lot of people

working on these

applications not their core Str

I see a lot of people either data

scientists data Engineers uh machine

learning Engineers enter into this field

and you really want to consider all of

the software development best practices

when you're going to put applications

like this into your production also for

people just in general new to the space

new to AI new to software

development these are all things that

you you have no idea about and I say

that because I still am like literally a

noob to most of these things and I've

been doing this this for 5 years already

so make sure that you understand what

you don't know and partner up with

people that do know that stuff in order

to make sure you can just provide

reliable solutions for your clients all

right and then let's get into an example

architecture diagram of a project that

we're literally working on right now and

I don't see a lot of people talking

about this and I'm just going to give it

away either way but this is I think in

my opinion such a cool such a nice

blueprint for lots of generative AI

solutions that that you can build right

now this the only thing that that's

missing is that this doesn't have a

front end that's because we're

integrating with another application

that we're uh getting data from and

sending data through but other than that

this is a really scalable robust and

secure framework that that we're working

with right now so high level it's it's

mostly just a a summary of of everything

that that I've discussed in part two in

part three of this video but for this

particular solution we're connecting

with an existing application so we're

using their API and we're using HTTP

triggers whenever something happens in

the system in that application we can

set up uh a trigger and it hits our back

end so it sends it sends a request to

our fast API backend and from there we

get all the data and that is where the

the pipeline is triggered so we use

queuing mechanism uh in here and we use

Reddit as an intermediate step I think

that's the only thing that I didn't

mention in uh in this dock in the video

in the document and through redis we

sent it to to celery to put it into a

queue we can also visualize that using

flow and then we set uh the output is

then uh put into the database so in the

postc SQL database so this is how we lck

internally all of the data that we're

using uh later for referencing for

monitoring and we enrich the data as

well but the the back end and the Q

worker that is where all the the magic

is happening in terms of the processing

so we take that data we we take it

through a data processing pipeline where

we perform various data enrichments and

also uh multiple large language model

queries in between to then finally get a

desired output and that output is sent

back into the system that we're dealing

with so this is a architecture that you

can copy for for lots of use cases where

you integrate AI into a current system

which is a really good use case now next

to that here we have a separate Resource

Group where we also have l view so we

talked about that then we also this is

the production version we also have the

test version which is a copy a literal

copy of what we have running in

production the only difference is that

it cannot send information back to the

source application and then the the cool

thing about this where when it comes to

security we put so we host all of this

in Microsoft Azure and we put it into uh

we created an Azure virtual Network and

then created the VPN Gateway and we only

allow incoming request through this VPN

Gateway and that means when we as

developers want to work with this we

with our laptops can connect with this

VPN Gateway and that is the only way to

connect with this and connect with these

resources anything else will be blocked

by default so this is how you can

entirely shut off your application so no

one can reach it because uh we're

dealing with sensitive information in

this uh in this database in this whole

system and we don't want uh that to be

interrupted of course and then here we

make sure we have the uh we have some

some type of multip multiactor

authentication so we send the request to

our back end but we also have an

authentication through a header in there

that we implement it so this is a really

cool in my opinion an architecture and

not all props to me this is really the

the software engineer that that I'm

currently working with that has

engineered this for the project that

we're working on so that concludes this

video so I hope this was helpful it's a

pretty long one I really went in depth

talking about how what we currently do

inside our company data Lumina so if you

found this video helpful please leave a

like and also consider subscribing if

you want to learn more about

uh what we're doing how we're helping

our clients or want to learn more about

freelancing how to get started with that

so make sure to subscribe if that sounds

like you and then for now make sure to

watch this video next that's the video

where I dive deeper into the client

acquisition part how you can find your

own clients how you can get started with

tools resources books that you can dive

into so make sure to check that out next

and then I'll see you in the next one