The Truth About Building AI Startups Today

Summary

在这段播客《光锥》中，Y Combinator的合伙人Gary、Jared、Harge和Diana探讨了人工智能（AI）如何渗透到社会的各个层面，并对初创企业产生影响。他们讨论了AI在初创企业中的应用，包括大型语言模型（LLM）的兴起，以及这些技术如何被用于自动化、开发工具和特定领域的定制化模型。此外，他们还讨论了AI领域的伦理、隐私和安全性问题，以及开源AI的重要性。最后，他们鼓励创业者寻找具体的AI应用场景，避免陷入过于泛泛的想法，并利用AI技术重新构想现有的软件解决方案。

Takeaways

• 🚀 AI技术正迅速渗透到社会的各个领域，包括商业交易和计算机使用，正在创造新的创业机会。
• 🌟 Y Combinator（YC）目前资助的初创公司中有近50%涉及大型语言模型，显示出AI领域的热度。
• 🎓 许多大学生和年轻的创始人正在退出学校，投身于AI领域的创业，因为这是一个一生一次的机会。
• 🛠️ 开发者工具，特别是用于提示工程的工具，正在兴起，因为它们帮助开发者更有效地使用大型语言模型。
• 🔍 工作流程自动化是AI应用的一个热点，特别是在那些涉及重复性任务的领域，如搜索信息或填写表格。
• 📈 即使是看似平凡的业务，如自动化政府合同搜索，也可能成为巨大的商机。
• 💡 无聊或平凡的业务背后可能隐藏着巨大的商业价值，深入挖掘可以发现宝藏。
• 🚧 AI领域的“陷阱”想法可能会吸引许多创始人，但最终难以实现真正的产品市场契合。
• 🔒 数据隐私和安全性是AI应用中的一个重要问题，特别是当涉及到使用私有数据集进行微调时。
• 🛑 微调和定制化AI模型可能比通用模型更有优势，特别是在特定领域，如医疗或金融科技。
• 🌐 开源AI模型的使用和定制化是构建新创业公司的机会之一，但需要超越仅仅提供更便宜的服务。

Q & A

什么是'The Light Cone'播客的主题？

The Light Cone播客的主题是技术，特别是人工智能的过去和未来。播客由Y Combinator的合伙人Gary, Jared, Harge和Diana主持，他们讨论了AI在社会各个方面的渗透，以及他们如何在工作中与最好的创业者合作。

Y Combinator在2023年夏季资助了多少使用大型语言模型的公司？

在2023年夏季，Y Combinator资助的公司中有接近50%使用了大型语言模型。

为什么许多创业者选择在AI领域创业，即使Y Combinator并没有特别偏好AI公司？

许多创业者选择在AI领域创业是因为这是他们认为有高潜力建立大型公司的领域。这不是因为Y Combinator有特别偏好，而是创业者们自己对AI领域的兴趣和信念。

为什么越来越多的创业者选择在大学中途退学来从事AI工作？

许多创业者认为AI领域可能是一生一次的机会，他们不想错过。此外，由于AI是一个新兴领域，即使是大学生也没有比他们更多的经验，因此他们认为现在是开始的好时机。

开发者工具在AI领域的重要性是什么？

开发者工具在AI领域非常重要，因为它们可以帮助开发者更有效地进行提示工程(prompt engineering)，测试他们的提示(prompts)，并查看第二顺序效应。这些工具对于构建和改进AI系统至关重要。

为什么在AI领域，看似平凡的工作自动化实际上可能是巨大的商机？

在AI领域，许多看似平凡的工作自动化实际上是巨大的商机，因为这些工作往往是重复性的，涉及大量的信息处理，而这正是大型语言模型(LLMs)擅长的。自动化这些任务可以显著提高效率和生产力。

为什么说'无聊'的业务可能实际上是一个惊人的业务？

根据Paul Graham的一篇文章，'无聊'的业务可能实际上是一个惊人的业务，因为它们可能解决了一个非常具体和实际的问题。深入挖掘并解决一个具体问题往往能够发现意想不到的价值。

什么是AI tarpit，为什么它对创业者来说是一个陷阱？

AI tarpit是一种表面上看起来很吸引人的创业想法，但实际上一旦深入其中，就会发现它并不是一个好的创业想法。许多创业者会被这种表面的吸引力所吸引，但最终可能会陷入其中，难以自拔。

为什么说大型语言模型在特定领域可能不如定制的小型模型有效？

大型语言模型虽然在广泛的任务上表现出色，但在特定领域，定制的小型模型可能因为针对性训练而表现得更好。这是因为它们可以专注于特定领域的词汇和任务，从而提供更精确和有效的结果。

为什么说AI的发展可能需要开源和公平的技术访问？

开源和公平的技术访问可以确保不仅仅是最大的公司拥有最强大的AI能力。这样可以防止技术垄断，确保所有消费者都能从底层获得相同的技术，从而防止潜在的滥用和不平等。

为什么AI领域的研究者现在更倾向于创业？

AI领域的研究者现在更倾向于创业，因为他们看到了像GPT这样的技术如何改变世界，并意识到他们的研究成果可以转化为具有巨大潜力的商业机会。

为什么说AI的发展可能会带来新的网络安全挑战？

随着AI技术的发展，特别是大型语言模型的使用，可能会出现新的网络安全挑战，比如数据隐私问题和模型的安全性。这就需要新的网络安全技术和解决方案来保护系统免受攻击。

为什么说AI的发展可能会让SaaS产品看起来像是'数据库前端'？

随着AI技术的发展，SaaS产品可能会被看作是数据库的前端，因为AI可以自动化许多任务，使得用户界面和用户体验变得更加简单和直观，就像早期的数据库应用一样。

为什么说AI的发展可能会让一些创业想法看起来像是'GPT rappers'？

'GPT rappers'是指那些在大型语言模型之上构建的创业想法，它们可能看起来很有吸引力，但实际上可能很容易被更高级的AI技术所取代。这个术语反映了一些创业想法可能只是对现有技术的简单应用，而不是真正的创新。

为什么说AI的发展可能会让一些创业想法变成'tarpet ideas'？

'Tarpet ideas'是指那些看起来很有吸引力但实际上很难成功的创业想法。在AI领域，由于技术的快速发展和变化，一些看似有前景的创业想法可能会很快变得过时或被更先进的技术所取代。

为什么说AI的发展可能会让一些创业想法变成'数据库前端'？

随着AI技术的发展，许多创业想法可能会变成'数据库前端'，因为AI可以自动化许多任务，使得用户界面和用户体验变得更加简单和直观，就像早期的数据库应用一样。

Outlines

00:00

🤖 AI技术与创业机遇

本段落讨论了如何区分有潜力成为大型企业基础的创意与可能被高级AI技术如GPT-5所取代的点子。强调了即使是看似乏味的点子也可能成为出色的商业模式。提到了Y Combinator（简称YC）目前对AI公司的投资趋势，以及AI技术如何渗透到社会的各个层面。此外，还提到了创业者对大型语言模型的兴趣，以及YC如何根据创业者的兴趣而非特定技术领域进行投资。

05:01

🛠️ 工作流程自动化与AI应用

这一段探讨了AI在工作流程自动化中的应用，特别是那些涉及重复性任务的领域，如搜索信息或填写表格等。提到了YC对此类应用的兴趣，以及创始人在寻找创业点子时可以考虑这一领域。通过Sweet Spot公司的案例，说明了如何将AI应用于政府合同搜索和提交提案的自动化，展示了即使是看似无聊的点子也可能具有巨大的商业潜力。

10:01

🚀 AI创业的陷阱与机遇

本段讨论了AI创业中的一些常见陷阱，如AI副驾驶（AI co-pilot）的概念，以及如何避免陷入这些表面上吸引人但实际上难以成功的创业点子。强调了专注于具体用例和解决具体问题的重要性，以及如何通过提供定制化的服务来满足特定行业的需求。同时，提到了YC在2023年夏季投资的公司中，有近50%涉及大型语言模型。

15:04

🔒 数据隐私与AI模型的定制化

这一段讨论了数据隐私问题，以及企业如何通过定制化AI模型来满足特定行业的需求，特别是在金融科技和医疗保健等领域。提到了Credle公司作为例子，展示了如何通过定制化AI模型来满足特定数据集的需求。同时，还讨论了数据隐私的担忧，以及如何通过技术手段保护私有数据不被泄露。

20:04

🛑 AI技术的发展与市场适应

本段讨论了AI技术如何快速适应市场，以及如何通过定制化和特定领域训练的模型来提供更好的服务。提到了如何使用旧版本的GPT模型来满足特定领域的需求，以及如何通过定制化模型来提高特定任务的性能。还讨论了AI技术如何帮助编程工作流程，以及如何通过AI辅助工具来提高开发效率。

25:06

🌐 AI技术的普及与开源运动

这一段讨论了AI技术的普及，以及开源AI对于确保技术公平性和可访问性的重要性。提到了AI技术如何被大型公司所主导，以及如何通过开源运动来确保所有人都能访问到先进的AI技术。还讨论了AI伦理和监管问题，以及如何在AI领域建立新的公司和创业机会。

30:07

🔄 AI技术的新周期与创新机遇

本段讨论了AI技术如何标志着一个新的创新周期的开始，以及技术专家和研究人员如何在这个周期中发挥关键作用。提到了YC如何回归其根源，专注于资助那些在新技术上工作的创业者。同时，还讨论了AI技术如何被市场低估，以及如何在这个领域寻找真正的创业机会。

Keywords

💡人工智能

人工智能（AI）是指由人制造出来的机器或软件系统所表现出来的智能，这种智能能够执行通常需要人类智能才能完成的任务。在视频中，人工智能是讨论的中心主题，特别是在创建大型语言模型（LLMs）和它们对创业公司的影响方面。例如，视频中提到了AI在自动化工作流程、改善用户界面和创建新的商业机会方面的潜力。

💡大型语言模型

大型语言模型（LLMs）是指能够处理和生成自然语言文本的复杂计算模型。视频中提到，LLMs正在成为许多创业公司的基础技术，它们被用于创建自动化工具、改进用户界面和开发新的应用程序。例如，视频讨论了LLMs在自动化政府合同搜索和提交提案方面的应用。

💡创业公司

创业公司是指刚开始运营的商业实体，通常寻求通过创新产品或服务来实现快速增长。视频中，创业公司是AI技术应用的主要受益者，许多创业公司正在利用LLMs来开发新的解决方案，如自动化工具和改进的用户体验。

💡Y Combinator

Y Combinator（简称YC）是一家知名的创业加速器，它为创业公司提供资金支持和指导。视频中提到YC作为合作伙伴，他们与世界上一些最优秀的创始人合作，并且正在见证AI技术如何影响创业生态系统。

💡技术创业

技术创业是指基于技术创新来创建新公司或产品的过程。视频中讨论了技术创业在当前AI浪潮中的重要性，许多创始人正在利用AI技术来解决具体问题，并创建具有巨大增长潜力的公司。

💡自动化

自动化是指使用机器、软件或其他技术来执行通常由人类完成的任务。视频中提到自动化在多个领域中的应用，如自动化政府合同搜索和提交，以及自动化营销文案的生成。

💡数据隐私

数据隐私涉及保护个人信息不被未经授权的访问、使用或披露。视频中讨论了在使用AI和LLMs时，企业对数据隐私的担忧，特别是关于将敏感数据集提供给第三方服务提供商的问题。

💡开源模型

开源模型是指源代码或数据集可供公众访问和使用的软件或技术模型。视频中提到了开源模型在AI领域的应用，以及它们如何被定制和微调以适应特定的业务需求。

💡微调

微调是指对预先训练好的机器学习模型进行额外训练，以适应特定的任务或数据集。视频中提到了微调在AI领域的应用，特别是在为特定行业或业务需求定制大型语言模型时。

💡多模态AI

多模态AI指的是能够处理和理解多种类型数据（如文本、图像、声音等）的人工智能系统。视频中提到了多模态AI作为AI领域的一个新兴趋势，它可能会在未来带来新的创业机会和应用场景。

💡AI伦理和监管

AI伦理和监管涉及确保人工智能系统的开发和使用符合道德标准和法律规定。视频中讨论了AI伦理和监管的重要性，特别是在AI技术快速发展和广泛应用的背景下。

💡技术创业的新时代

技术创业的新时代是指由新兴技术如AI推动的创业活动的新阶段。视频中提到，我们正处于这样一个新时代，其中技术创业者有机会利用AI来解决复杂问题，并创造具有重大影响力的公司。

Highlights

Y Combinator合伙人讨论了如何区分可能成为价值十亿美元公司基础的想法与可能被GPT 5所取代的想法。

大语言模型（LLM）在2023年夏季的YC孵化项目中占据了接近50%的比例。

智能创始人选择YC投资的AI项目，并非因为YC偏爱AI，而是他们认为这是建立大型公司的高风险机会。

许多创始人因为AI技术的兴起而选择辍学创业，认为这是一个千载难逢的机会。

开发者工具在提示工程（prompt engineering）方面的应用正在增加，特别是对于大学生和年轻创始人。

工作流程自动化是AI技术应用的一个热点，尤其是在替代重复性任务方面。

YC合伙人分享了Sweet Spot公司如何从食品车订购应用转型为利用LLM自动化搜索政府合同的案例。

讨论了“无聊”的业务可能实际上是极好的商业机会，引用了PG的“有泥就有铜”的观点。

AI领域的“塔皮特”（tarpet）想法，即看似吸引人但实际上难以成功的创业想法。

AI辅助驾驶（co-pilot）的概念可能太早，尚未找到产品市场契合点。

讨论了定制化开源模型服务的需求，以及如何超越仅仅提供低成本替代品。

数据隐私成为企业考虑使用AI服务的重要因素，促进了新的网络安全解决方案的发展。

特定领域定制的小型模型可能比通用大型模型表现更好，例如SQL查询解析。

YC合伙人分享了如何使用AI重新构想现有软件，例如Salesforce，以利用AI的能力。

讨论了AI语音代理在小企业中的应用，例如作为接待员自动安排预约。

对开源AI的支持，以确保技术不被单一公司垄断，保证技术的普及和公平性。

AI研究者对创业的兴趣增加，许多基于研究论文的公司成立，反映了技术与商业的结合。

YC合伙人强调了在AI时代，寻找具体需求和定制化解决方案的重要性，避免陷入通用但无效的AI应用。

Transcripts

how would you differentiate between an

idea that could be a great foundation

for a billion doll company and an idea

that is likely to get run over by GPT 5

something that's boring might actually

be an incredible business but why is

that yeah let's talk about GPT rappers

are people worried about giving these

data sets to open AI all these AI agents

are passing the touring test I mean this

is why I think the chat interface is

wrong you want to do something in AI

like this is a good place to like look

into big generational companies are

getting built as we speak great startup

ideas just lying on the ground you'd

like trip over them this might actually

be like a once- in a lifetime

opportunity and I I think I actually

agree what a time to be

[Music]

alive welcome to the very first episode

of the light cone I'm Gary this is Jared

Harge and Diana and we're group Partners

at Y combinator and we get to work with

some of the best Founders in the world

Jared why are we calling it The Light

cone well in special relativity the

light cone is the path that light takes

from a flash of light you can imagine a

flash of light and it spreads out in a

cone shape and in special relativity you

think about it spreading out in a cone

both in the future but also in the past

and in this podcast we are here in the

present but we are going to talk about

both the past and future of technology

so that's how we came up with the name

and one of the things that we're all

seeing is the encroachment of AI into

almost every piece of uh Society at this

point you know every business

transaction every uh thing that we sort

of use with computers uh suddenly a new

burst of technology is sort of entering

everything we're doing and we're seeing

it in the startups that we're funding

which is why we're so excited about it I

think you know what what's the

percentage of companies you've backed

right now that have large language

models I think for summer 23 was close

to 50% of the batch and it's pretty

interesting like I think a lot of people

like see that number and they think oh

YC must have funded so many AI companies

because we have this thesis about Ai and

like it's just easier to get into YC if

you're an AI company because we just

like love funding AI companies and it's

funny to us because we know how that's

not true and yet that's probably what

like 90 that's probably how 90 plus per

of people actually think YC Works how

does Howes how's it actually work can we

tell people like how it actually works I

actually think it's interesting the

smart Founders apply to us with what

they want to work on and we fund the

smart Founders like irrespective of what

they want to work on actually and

exactly and so the fact that half the

batch is working on AI says something

much more interesting than just the YC

Partners think AI is cool it's an

emergent phenomenon of what the the

smart Founders want to work on right now

is like where do they think there's the

high beta to build the largest company

and I think the most ambitious and

smartest Founders are going after this

because it's definitely I think the

exciting thing about right now with AI I

think it's like real there's been a lot

of waves for AI and multiple AI Winters

but this one actually gbt 3.5 and then

four blew out of the water a lot of task

and it impressed a lot of smart people

when a lot of smart people start paying

attention and building in this current

idea mace I think big generational

companies are getting built as we speak

one thing I'm seeing that's interesting

is I feel like a lot um a lot more

Founders are dropping out of college to

start working on AI because they don't

there's a f off yeah there's like an

actual like and usually it's so funny my

my interview question is almost always

like what's the rush like why do you

want to drop out of college like why

don't you just like graduate because it

makes a lot more sense to graduate and

then do a startup um and the reply is

usually like well like this might

actually be like a once in A- lifetime

opportunity and I I think I actually

agree and and the other cool thing is

that this is an opportunity where

college students are particularly well

like young Founders are particularly

well positioned to work in it because

nobody has like like there's no one

walking around with like four years of

LM experience so like everyone is

starting from the same playing field and

so if you can learn fast you're going to

be at the same level as everybody else

that's right and you know one an area

I've seen that come to play is like

developer tools for prompt engineering

I've been seeing like these sorts of

tools are getting uptick it's like

ability to like chain together different

prompts and test your prompts and see

like the second order effects um and

actually a lot of college students are

the people who are just like playing

around with like prompting models and

seeing what comes out and it's a really

easy startup idea for them to like just

build the tools that they want and like

the tools that they want are literally

setting like the standard for what every

developer should want like I know a lot

of the headlines are all around like AGI

and all of the fancy stuff and then the

really cool demos of like multimodal AI

like AI generated video and and this

kind of stuff the stuff that I've seen

in the batches actually taking off is a

little bit more mundane like it's um I

probably say a lot of it sort like

workflow automation like um it's finding

things where there was like a human

doing some repetitive task usually

involved like searching for things or

filling out forms and then using like

llms to replace that it feels very

obvious to us the people who work at YC

that this is an amazing opportunity

there's so many jobs in the world that

are basically very mundane information

processing typically stuff that's hidden

in some back office somewhere where

there's somebody who's just like reading

stuff and summarizing it re-entering it

from one system into a different system

and like a slightly different format and

it's such a perfect fit for llms LMS are

like perfect for this job and yet we

actually don't get that many

applications for people working on this

and there's a lot of Founders out there

who are searching for a great idea so if

you're out there and you're looking for

a great startup idea and you want to do

something in AI like this is a good

place to like look into I give you an

example so last patch had a company I

worked with called sweet spot and we

funded them the idea was something about

like food ordering from food trucks

something like random and they pivoted

immediately looking for a new idea and

the idea they found was um using llms to

automate searching for government

contracts to bid on and God such a good

idea yeah and submitting the proposals

that sounds so boring what could be more

boring than searching through like a

list of all the government contracts you

know how they found it is um exploring

startup ideas and then they realized one

of their friends his job was to work for

one of these like government contractors

and his whole day was just spent like

refreshing this government website um to

like find things and then submitted

proposals and they're like what like

that's like exactly that that's so

boring like wouldn't you like a tool

that did this for you yeah and they

launched and like pretty much straight

out of the gate got like um a pretty

decent amount of traction because

they're like opening up um the people

who who would actually do it like it

becomes easier to like find government

contracts to bid on when it's all

automated away and like software does it

for you you know obviously we all know

that you know something that's boring is

actually kind of awesome but why is that

that's like you know just off the bat

you know we have a sense that something

that's boring might actually be an

incredible business there's an old PG

essay where he talks about this and he

says um he he quotes a phrase where

there's muck there's brass it's like

it's as it's almost like Old English you

want to explain it har just means like

you can find treasure in surprising

places yeah and I think the cool thing

is you have to go deep and vertical and

solve a very concrete problem like some

of the problems with let's maybe talk

about AI tarpits what a tarpet idea is

is it's an idea that from the outside

looks really shiny and attractive it

looks like a great startup idea and so

lots of Founders go and they start

working on it and then you realize once

you're in it that it's actually not a

good startup idea but but by the time

you're there you're like stuck in it and

so it just attracts founder after

founder and they just get stuck in the

tarpet idea and we see this a lot at YC

because we see all these applications

and so it's really obvious to us when

like 500 people apply to a YC bat for

the same idea but they don't know that

499 other Founders are also stuck in the

same tarpet what's tricky I think about

topet ideas for AI is like we know

something's that top it idea in

hindsight once like enough people have

been stuck in it so with AI it's so new

we don't know yet so I have a couple

that I'm actually like Keen to get

your's thoughts on um a very common one

is AI co-pilot so it's like hey I'm

going to make it easy for um people to

like build an AI co-pilot for their

product or or service it's it's really

unusual type of phenomenon where there's

so much interest from potential

customers to like want a co-pilot that

it's actually quite easy to start

getting getting like inbound leads if

you pitch this and if it's even easy to

get people to pay you money up front but

what's really hard is to get them to

actually like use the co-pilot because

they don't actually know what they want

it for like they just heard that AI

co-pilots might be changing the feature

of software so we should have an AI

co-pilot but they don't actually know

what their customers will use it for I

think for me and maybe I just have a uh

a mental block around chat interfaces

but I've never been that big a fan of

chat because it puts so much of the

emphasis on the user knowing how to

speak to a computer and you know while

in the next five or 10 years I think we

will all get far more used to using it

that way um I think the the lwh hanging

fruit right now is just using the large

language model to actually do the sort

of knowledge work that a human being

could do and then package it into the UI

that you know whether it's a mobile app

or a web app that is just familiar like

sort of what people use to do their work

right now and it's you know basically

the llm is better used as sort of this

like I I mean it's almost like you know

this thing that's sprinkled in that you

know the software suddenly does

something really powerful but you don't

have to change the way you would want to

use the software as it is sort of like a

an example of a phenomenon that like I I

think we have seen in the past when like

some technology gets really hot and all

of a sudden like all these companies are

like they're being asked by people like

what's our AI strategy they're like oh

well we better get an AI strategy or

like with crypto there was like oh

everybody needed a blockchain strategy

and even before that it was like

everybody needed a mobile strategy for a

moment in time it's like easy to sell

them something that like placates their

desire to check some box but in the end

you've got to actually make it

successful for them like otherwise it's

not going to stick I agree and so like

perhaps with this AI co-pilot thing like

maybe it's too early to call like

perhaps they actually will find product

Market fit maybe with something that's

not a chap out UI like they'll like keep

iterating on the UI until they find

something that's an AI co-pilot people

actually want or maybe it's just going

to like fizzle it just like turns out

most people don't need an AI co-pilot

some of the advice I've been giving

those those specific companies is the

another old PG essay about if you if

you're trying to sell technology to

someone and they're not buying like see

if you can just build a competitor and

so it's like hey if you're trying to

sell like um

uh fintech company a co-pilot and

they're not buying it well like if you

are convinced they should have a

co-pilot like why don't you just like

build the company with the co-pilot as

the main experience and see if you can

out compete them or not I like that that

I like that I think getting people to

focus on the use case I think the

problem is the whole thing with um kind

of the Gold Rush people selling more the

shovels and the tools and even then in

this case it is a bit of that but a lot

of people aren't digging gold yet like

the reality is this is such a new

technology and even the end applications

that apply AI the reality is there so

early they don't have product Market

fits so it's sort of bit of a the blind

leading the blind in here it's like what

do I even know what the pattern is for

copilot I mean it sounds cool just to

join the cool kid Club of we're doing Ai

and we're going to check mark So I think

that's the danger for a lot of these uh

startup it's like it seems that they're

getting traction as you mentioned but

then when you we poke them closer is

anyone actually using you what are the

actual use case and then the founders

come back and they startare a blank at

us oh but look at all the sign up look

at the revenue but then they're not

really using your product I mean we're

seeing even the second order effects

right so a bunch of us are funding uh

Dev tools companies that sell to AI

companies and they're selling tooling

but then they might you know they might

sell an Enterprise contract to someone

who also Upstream has a Fortune 00 that

said that they'd pay $100,000 a year for

that contract and then 6 to n months

later that you know Fortune 100 went

back to the incumbent uh you know some

other leading you know IBM Salesforce

like something like that um because they

ended up adding large language model

technology to what they they were doing

and people just switched back and

suddenly the dev Tool Company suddenly

realizes oh I had five contracts but

three of them went away because my

customer actually their customer so it's

actually like sort of remarkable how

fast this is evolving you know right now

in 2024 a specific type of idea I'm

curious to get thoughts on here as well

is um offering like fine-tuning open

source models sort of as a as a service

broadly like that's a very popular idea

I think over the course of 2023 here's

what I've seen so like why do people

want like why is there any demand for a

fine-tuned like open source model at all

um it tends to be initially I think the

Big Driver was cost like open AI like

chat GPT was expensive and people wanted

a um cheaper version of it and so I

think it was very easy to get customers

with the pitch of hey like we can f tune

an open source model and it's just going

to be much cheaper what I think a bunch

of the companies in space are seeing is

that like that's not enough to keep the

customers especially because like open a

like the cost of all of the models just

going down and that's going to keep

happening with the

open AI has a plan for all of those so

there's something more that all these

fine-tuning companies need to do yeah it

has be better not just cheaper I think

where is exactly that where I think is

having more legs is when these companies

need to customize it to private data

sets so you have the open General big

foundation model but then you have to

tune it up

to specific data sets that for example a

healthcare or fintech can't give out can

give out and they don't have the team of

um experts to do it so I think the one

company that I think Brad worked with

was credle that kind of was doing that

what are you seeing about like so the

concern around data privacy is another

big reason like are you seeing that as

being enough like are people worried

about giving these data sets to open AI

it's really interesting I mean whenever

you have something so new like this it's

actually um sort of resets the clock on

the competitive landscape again so

you know you almost can expect all the

same things will happen again um you

know just as 10 15 years ago Cloud was

brand new and then you had Cloud cyber

security and Cloud strike and all these

companies sort of come out um you know

we're seeing the first wave of cyber

security companies you're like prompt

armor so they sort of wrap your API

calls and uh what they actually have

figured out is that for a lot of large

language models if you do any sort of

fine-tuning or training with private

data you can actually just speak to the

model

and get it to spit out your private data

again and they have a solution that

stops IT so it's so interesting because

you know it's entirely possible you know

they're basically creating a new

industry again um of cyber security for

llms sort of in the same way that cloud

opened up that space and created cyber

security for the cloud yeah I definitely

think that whole world of controlling

within an Enterprise in particular like

controlling who has access to like which

llm has access to like what data and who

has permission is like a really ripe

space for building interesting software

I think the other exciting area that a

lot of the tools are getting built is

getting more this is like a step further

fine-tuning but more purpose

trained models that are smaller so take

a for instance a llama and getting those

to run locally in machines for inference

and when you customize some train on a

specific domain and Target data is going

to perform better than the general model

The General model was kind of trained on

all of the human language for all of the

task but if you wanted to build like the

best let's say um language model for

parsing SQL queries you would then parse

very specifically just a set for SQL

quer and I think some of those that are

interesting companies that we funded is

like AMA that you funded that's trying

to make the development process for

running all of these locally a lot

faster and I think we're also funding

some of these that are custom for coding

the thing that was surprised learning

from some of the startups that are

building um coder type of uh co- Pilots

which I think is is a use case that's

working out making a lot of the workflow

for programming a lot faster it's kind

of like autocomplete and co-pilot type

of thing they're training on older

models of a GPT they don't even need the

newest one and then I asked like why is

that and even for like one of the

companies who funded last batch

metalware for Hardware they're not using

the stateof the AR model like the older

GPT I forget which one was like the

older 2.5 or three was sufficient and

actually creating good enough results

because the vocabulary for a specific

domain for Hardware or software is a lot

smaller than the human language so this

is other world where the open model

that's customized I think is going to

win and compete versus the big one for

specific domains so there lots of

companies with this yeah that's what uh

Toby loty from uh shop actually still

dabbles with the stuff I think he

actually built the uh internal co-pilot

for Shopify and what he was saying is

the best way to use whatever gp4 or the

you know latest Clos Source models that

are most expensive and have the most

parameters uh just think of it as a

prototyping tool anything you do with

those prompts you can get your own model

to do with a little bit more training

it's kind of like uh when people build

Hardware you have the analogy of uh

prototyping with fpga

which are very expensive right and then

when you have the right architecture for

Hardware then you do the circuit path

and actually do the custom s so so right

now for some of these tasks the large

language model is sort of like your

fpga whatever GPT 4 and then when you

customize it you do like the super

efficient one coding path for I don't

know Shopify for coding assistance and

Hardware software Etc that becomes your

so that you train and customize which is

cool I think that patterns emerging it's

like as I hear you talk about that

what's I just think it's just like so

many different startups that could be

built it just feels like we've never had

this moment at least I didn't feel like

I've never experienced a moment where

there's just so many potential startup

ideas to be built like all that ones

yeah there there absolutely hasn't in we

we definitely saw this in the last batch

with all the pivoting companies oh yes

people don't always realize this but

like many of the companies get into YC

within a month after we fund them

they're looking for a new idea cuz the

old thing didn't didn't work or they

lost interest in it or something and

it's normally like not actually that

easy to find a great startup idea for a

team to work on but man was it easy last

summer God it was just just like great

startup ideas just lying on the ground

you'd like trip over them yeah that was

a fast I think you actually had a tweet

about it that was one pretty uh viral

that talked about this is the batch the

batch ever in your whole career working

at YC where Founders got to good ideas

the fastest ever and hard has been here

even even longer yeah know it definitely

feels unique I've never had so many

successful pivots yeah and Gary to your

point about the chat gbt rapper I think

back like I feel like that Meme really

came out like just about a year ago yeah

let's talk about GPT rappers yeah like

like I feel like the first sort of group

of ideas I saw in the batch were all

generative AI ideas built on Chop top of

chat gbt so was stuff like hey like

automate your marketing copy or automate

like your creative content or something

like that and that term got thrown out

oh these things are all just like

rappers on top of chat GPT and um open

AI is going to like take all of like

it's just going to build all of these

things and they were going to release

their App Store and like it's just going

to take all the value and these things

will die of the mem all of all of SAS

software is just my sequel rappers

exactly I think this is a great analogy

you can think about any SAS product as

basically a database rapper like you

could imagine like negging any SAS

product CU like the first version of a

sass prod it's basically just a crud app

and just like you took like my SQL then

you like built like a website on top of

it and I think people are going to look

back on this term GPT GPT rapper like

similarly how we think of like how we

would look at the term database rapper

which just seems like silly I mean this

is why I think the chat interface is

wrong like I actually think there is

value acur to really great ux like good

copy good um you know interaction design

information hierarchy uh you know being

able to approach a product and say like

this is the job to be done and for for

users to come in just sort of naturally

understand what to do like there is a

craft to building software that is

timeless and that sort of transcends

whether or not you're using a large

language model and so you know that that

I think is what I mean by you know these

things are not you know SAS software is

not uh a MySQL rapper well here'd be a

question I'd be interested in in in

everyone's thoughts on suppose you're a

new founder and you really want to build

a

company and you want to do something on

top of

LMS how would you differentiate between

an idea that could be a great foundation

for a billion dollar company and an idea

that is likely to get run over by gbt 5

and is probably like not a good starting

point I think if a Founder is working on

something too General and not solving a

specific need for a user they can

actually go talk to another use case so

I I worry about the ones that are too

generic generic and building going

after some kind of

abstract it will solve all the things

yeah if it's like hey like throw your

data in here and we'll do like

automations on top of it like for

everything that's probably hard to

compete with whatever one of the

foundation models might offer but if

it's like hey we are give us like your

sales log data and will like um spit

back like suggested next actions like

you can like for sales people to make

them better at sales that's probably

going to work better or give us all your

compliance checklist to pass Hippa

compliance and process that it's like

that's very specific and lots of

business logic or give us all of your

data

for processing government forms right

yeah so a lot of custom business logic

so the same thing with the SAS era a lot

of the applications and how you build

applications in there there's always the

separation business logic and they crow

in a lot of architectures for these app

and a lot of the value of the company is

accured on that business logic that is

so custom per company and there's a

whole pattern of uh programming patterns

on how people separate those yeah gu as

this all goes multimodal this is going

to get really interesting so early days

but yeah we've seen companies work on

voice AI apps to be like a sales rep and

I think um it's an interesting example

of the kinds of ideas that might be

possible now with AI is where you take

something like a Salesforce and you try

and reimagine like what would Salesforce

do if it were started today with all the

power of AI what it almost certainly do

more than just be like a CRM right like

it would make like it would find who

your leads might be like maybe now it

can make the calls for you it could like

set them up like maybe it goes all the

way to start like implementing like the

first version of the product for them

like I think it's just like the scope of

software you can build with AI now is so

big I think that's another good way to

find ideas like look at software today

and reimagine it with the power of AI

today which you funded a number of

companies that effectively are AI voice

agents for small businesses because they

receive I don't know if you're like a

flower shop or a AC repair man in the

middle of U the US there's a lot of

calls for you to schedule and you don't

have a lot of stuff automated and

there's these YC companies that are

using that building these AI voice

agents to basically be the

receptionist I know one of our partners

Paul buight is quite worried about this

actually he's worried about there's

going to be a world of just s like all

these AI agents that are out trying to

do malicious things and that we're going

to need like our own like good defensive

AI agents out there making sure we don't

get scammed out of all of our money I

mean this is actually why I'm so uh an

advocate for open source AI because

these things are sort of real

considerations um you know can you

imagine there only being one hyperd

dominant AGI and it's totally close

Source it's owned by one company and uh

you know it's only available to the

highest bidder and uh you know imagine

you being uh you know someone who just

had to go to the doctor and uh on the

other end of it is uh some health

insurance company that uh you know

bought the bought access and blocked it

out from everyone else and you know you

getting on the phone you're not able to

sort of navigate or go against the sort

of you know impenetrable AGI that is

able to sort of get around anything that

you know your side might throw at it

like we actually want you know some form

of actually Equity at the AI level like

we actually want uh you know not merely

the biggest companies to own the most

capable AIS we want all consumers to be

able to have from the bottom up uh the

same access to that same technology and

that's uh you know the best insurance

against tyranny

certain that's actually what a lot of uh

also not just Founders but smartest

researchers who are really at The

Cutting Edge is I went to near IPS this

past December which was incredible to

see the energy in there the conference

has grown so much I think it like over

10,000 attendees there were 3,000 papers

more than 3,000 papers accepted and I

think um back in 2017 there was only

around 600 papers when I went back in

2010 it was was just in a ski lodge and

maybe like a 100 papers it's crazy the

kind of exponential growth and one of

the big topics of Interest was a lot

around AI ethics and Regulation and how

do we measure that so that that was

interesting um but the thing that's

different about typically that was

interesting in this conference is the

amount of interest from researchers

wanting to start companies too one

interesting data point is um a lot of

this era with GPT came about from from

One Foundation paper is all attention

you can need it was this paper that got

released got launched in a New York IPS

back in 2017 it was a team at Google who

was trying to figure out how to make a

machine translation between

languages more cheap because the English

translation to any language was actually

pretty good but if you wanted to do I

don't know German to Japanese there was

not enough data so they figur out this

way to compress data which became the

Transformer models for GPT and it was

like groundbreaking and this is the

foundation for llms that paper came out

in

2017 and the fun fact I was just looking

this up out of all those author eight

authors seven of them start at different

companies and all of the companies in

total their rate their worth valuation

more than six

billion and now people are seeing oh

these like industry Pioneers did this

and it's creating this new crop of I

think Founders that I don't think would

have started because I talked to a lot

of AI researchers and I don't think they

wanted to be Founders and I got a l this

question how can I turn my paper into a

company which I think is cool because

this is like going back to the root of

um why I F funding hardcore technical

Founders and I think it's cool to see

that energy there so when we went and

host our event we uh I didn't plan and

it was like 3x over subscribed nice

standing room only huh yeah yeah it's

that sounds like really the new Homebrew

Computer Club so NPS in December yeah we

got to mark it on the calendar we'll

come back yep Diana I love your point

about how this is sort of like returning

YC to its roots it definitely felt that

way last summer because when YC got

started the internet was really new and

the people who were building stuff on

the internet were mostly technologist

because actually like pretty hard to

build websites back then and pretty hard

to build like good software and like as

building software and building websites

got commoditized a lot more people came

into the

space and this is a cool reversion back

to the like Origins where like the

people who are building the most

interesting stuff are like mostly really

hardcore like researchers and

technologists because there's actually

real new technology being invented it's

not just like innovating on business

models with like commoditized technology

and again just like every great

technology it's being dismissed right so

going back to like the chat gbt rapper

meme I actually think that was great for

YC because it meant we only got the

people who are like tune who could tune

that out and we just like hey like

either I'm just so interested in this

technology I don't care like what the

memes are or I'm just too busy building

it to pay attention to the meme on

Twitter which is also great but like I

feel like this has always been the case

right like Homebrew Computer Club like

PCS are like dismissed as like toys like

the internet is dismissed as a toy like

all all of these things so feels like

that moment again yeah there is a a

class

essay that I love that I saw off Hacker

News do you guys remember this it's

Geeks mops and

sociopaths in a subculture Evolution and

you know I think that that actually is

the one thing that's quite durable and

like keeps returning right it's always

the Geeks Who are going to be into the

tech no matter what they're on The

Cutting Edge you know uh I always think

of Steve wnc talking about like you know

we started Apple computer with no idea

that it would ever be a company like we

just wanted computers for ourselves and

our friends and so you know at some

point the you know sociopaths come along

and they start sort of uh monetizing the

people who you know come to the scene

and then the cycle returns and repeats

so that's why I like being at the

beginning of a new cycle and clearly AI

is exactly that so don't don't count it

out don't write it off it's one of the

most interesting things that are is

happening out there um but you know

there are clearly things to be careful

of like don't be uh attracted to the new

shiny thing uh instead look for the muck

because where there's muuk there's brass

so that might be a great place to call

it for the very first episode of the

light cone we'll see you next

[Music]

time