Inside OpenAI [Entire Talk]

Summary

本次访谈中，OpenAI联合创始人兼首席科学家Ilya Sutskever讨论了人工智能的发展，特别是大型语言模型GPT-3的创新及其对社会的影响。Ilya分享了他对AI未来的看法，包括对深度学习、专业训练与通用训练的见解，以及OpenAI从非营利组织转变为有限利润公司的背后逻辑。他还谈到了AI的伦理问题，以及如何平衡技术进步与全球公民责任。Ilya鼓励对AI和创业感兴趣的学生，要信任直觉并利用自己独特的才能和视角。

Takeaways

• 🌟 伊利亚·苏茨凯弗是OpenAI的联合创始人兼首席科学家，对大型语言模型GPT-3及其前身产品ChatGPT的发展做出了重要贡献。
• 🎓 伊利亚出生于俄罗斯，成长于以色列，并在多伦多大学完成了数学学士和计算机科学硕士及博士学位。
• 🧠 伊利亚认为，人工智能的发展受到人类大脑神经元工作原理的启发，人工神经网络通过学习数据来提高预测能力。
• 📈 伊利亚讨论了人工智能的潜力，包括在特定领域如法律或医学中通过专业数据集训练来提高性能。
• 🚀 伊利亚提到，尽管目前人工智能在学习和适应方面与人类存在差异，但未来可能会达到或超越人类的学习速度。
• 🌐 OpenAI的使命是确保人工智能的发展惠及全人类，公司从非营利组织转变为有上限利润的公司，并与微软建立了合作关系。
• 💡 伊利亚强调，随着人工智能能力的增长，未来可能会出于安全考虑而不再开源这些模型。
• 📊 伊利亚认为，衡量OpenAI成功的关键是技术进步、研究执行情况以及对模型的控制和引导。
• 🔍 伊利亚提到，尽管OpenAI的某些方面是封闭的，但公司仍为学术研究提供了不同形式的模型访问权限。
• 🎭 伊利亚个人非常喜欢GPT-3的诗歌创作能力，他认为这是人工智能一个有趣且引人注目的特点。
• 🤖 伊利亚讨论了人工智能在团队中的集成如何提高生产力，尽管目前尚未对团队动态产生显著影响。
• 🌍 伊利亚表达了对全球人工智能监管和合理进步的看法，认为未来可能会有更多的政府监管来确保人工智能的安全和负责任的使用。

Q & A

Ilya的背景和教育经历是怎样的？

Ilya出生于俄罗斯，五岁时移居以色列，在以色列度过了他的成长岁月。他在以色列完成了大学前半部分的学习，之后转学到多伦多大学，获得了数学学士学位。之后，他在多伦多大学继续深造，获得了计算机科学的硕士和博士学位。

Ilya在OpenAI的角色是什么？

Ilya是OpenAI的联合创始人和首席科学家，致力于构建对全人类有益的人工通用智能。

Ilya对于大型语言模型的看法是什么？

Ilya认为大型语言模型是一种强大的技术，通过训练大型神经网络来预测文本中的下一个词，从而实现对人类语言的深入理解和生成。

Ilya如何看待人工智能的未来，特别是在学习速度和适应性方面？

Ilya认为虽然目前人工智能在学习和适应方面与人类有很大不同，但他不会押注反对人工智能在未来某个时刻达到或超过人类的学习速度和适应性。

Ilya对于意识或自我意识的看法是什么？

Ilya对意识问题持开放态度，他认为意识可能是一个程度问题，并且对于未来的人工智能系统，如果它们能够展现出与人类类似的意识特征，那将是非常值得思考的。

OpenAI的使命和转变是怎样的？

OpenAI的使命是确保人工通用智能的利益普惠全人类。它最初是一个非营利和开源组织，后来转变为有限责任公司，并与微软建立了紧密的合作关系。

Ilya如何看待OpenAI的开源与闭源的转变？

Ilya认为在人工智能的能力较低时，开源是有益的，但随着能力的提升，出于安全考虑，闭源可能变得必要。OpenAI的闭源决定是出于对人工智能能力增长的预期和对安全性的考虑。

Ilya对于OpenAI的盈利模式有何看法？

Ilya解释说OpenAI是一个有利润上限的公司，这意味着一旦对投资者的义务得到履行，OpenAI将再次成为一个非营利组织。他认为这种结构是独特的，并且有助于平衡公司的使命和经济需求。

Ilya对于人工智能在全球范围内的监管和伦理问题有何看法？

Ilya认为未来的世界需要明智的政府监管来确保人工智能的发展是可控和负责任的。同时，他也认为每个国家都有其在全球人工智能发展中的角色和责任。

Ilya对于想要进入AI领域的斯坦福学生有什么建议？

Ilya建议学生们应该发挥自己的独特优势和直觉，深入研究自己感兴趣的领域。他认为在研究和创业方面，独特的视角和创新思维都是非常重要的。

Ilya如何看待深度学习在未来5到10年的发展？

Ilya预计深度学习将继续取得进步，尽管通过扩大模型规模来获得进步的时代可能已经结束，但深度学习在许多层面上仍有改进的空间，这些改进将共同推动领域的发展。

Outlines

00:00

🌟 欢迎与介绍

本段介绍了斯坦福大学与YouTube社区合作的创业思想领袖研讨会，特别邀请了OpenAI的联合创始人兼首席科学家Ilia Sutskever。Ilia在人工智能领域有着杰出的贡献，是GPT-3等大型语言模型的基础思想者。他的个人背景和职业经历也被简要介绍，包括他如何从俄罗斯移居以色列，并在多伦多大学完成学业，最终成为OpenAI的关键人物。

05:01

🤖 大型语言模型技术

Ilia解释了大型语言模型的工作原理，包括人工神经网络的学习过程和反向传播算法。他强调了这些模型如何通过预测文本中的下一个词来训练，并指出这些模型的成功依赖于它们对“接下来会发生什么”的预测能力。他还提到了人类与机器学习方式的差异，以及如何通过大量数据训练来提高模型的性能。

10:02

🧠 人工智能与意识

Ilia探讨了人工智能与意识之间的关系，分享了他个人对意识的好奇和探索。他提出了一个未来可能进行的实验，即在不提及意识的情况下训练AI，然后观察AI是否能自行理解和表达意识这一概念。他还讨论了意识的度量问题，认为意识可能是一个程度问题，而不是简单的二元对立。

15:03

📈 OpenAI的使命与伦理

Ilia讨论了OpenAI的使命，即确保人工智能的利益普惠全人类，以及公司从非营利组织转变为有限责任公司的过程。他解释了与微软的合作关系，以及如何平衡公司的商业目标与对AI安全性的考虑。他还谈到了自己作为首席科学官的角色，以及他对公司发展方向的看法。

20:04

🌐 全球视角与AI的传播

Ilia讨论了AI技术在全球范围内传播的重要性，以及他作为世界公民的责任。他提到了与其他国家和地区的合作，以及如何在全球范围内推动AI技术的合理使用和监管。他还分享了自己对于AI技术未来发展的看法，包括深度学习的演进和专业领域内的应用。

25:06

🎯 OpenAI的未来方向

Ilia对OpenAI的未来方向提供了见解，包括公司可能成为人们直接使用的平台，或者作为其他应用背后的支持技术。他还讨论了ChatGPT在团队工作中的应用，以及它如何提高生产力和效率。最后，他分享了一些关于ChatGPT的有趣用途，如写诗和说唱，以及它如何被整合到OpenAI的团队工作中。

Keywords

💡人工智能

人工智能是指由人造系统所表现出来的智能行为。在视频中，Ilya 讨论了人工智能的发展，特别是大型语言模型和通用人工智能的研究。人工智能技术的进步对社会和经济有着深远的影响。

💡深度学习

深度学习是机器学习的一个子领域，它通过构建和训练多层神经网络来模拟人脑处理数据的方式。视频中讨论了深度学习的进步，以及它如何推动了像 GPT-3 这样的大型语言模型的发展。

💡神经网络

神经网络是一种模仿人脑神经元结构的计算模型，用于识别模式和处理复杂的数据。在视频中，Ilya 解释了神经网络是如何通过学习来预测文本中的下一个单词，从而实现对语言的理解和生成。

💡GPT-3

GPT-3（Generative Pre-trained Transformer 3）是一个由 OpenAI 开发的大型语言模型，它能够生成连贯的文本、回答问题、翻译语言等。视频中提到了 GPT-3 的发布以及它对世界产生的影响。

💡OpenAI

OpenAI 是一个致力于确保人工智能（AI）技术惠及全人类的研究实验室。视频中，Ilya 作为 OpenAI 的联合创始人和首席科学家，分享了该公司的使命和他在其中的角色。

💡技术创新

技术创新指的是创造或改进产品、服务或流程的新颖方法。视频中，Ilya 强调了他在人工智能领域的创新工作，包括他在深度学习和大型语言模型方面的贡献。

💡意识

意识通常指的是个体对自己思想、感觉和周围环境的感知和体验。视频中，Ilya 探讨了意识的本质以及它是否可能在人工智能系统中出现。

💡伦理

伦理涉及道德原则和行为准则，它指导个人和社会的行为。在视频中，Ilya 讨论了作为 OpenAI 首席科学家，他如何看待公司在人工智能发展中的伦理责任。

💡创业

创业是指创建新的业务或企业的过程，通常涉及创新、风险承担和资源整合。视频中，Ilya 作为一个成功的创业者，分享了他对有志于在人工智能和创业领域发展的斯坦福学生的建议。

💡数据集

数据集是用于训练机器学习模型的信息集合。在视频中，Ilya 讨论了使用特定数据集进行训练以提高人工智能在特定领域（如法律或医学）的性能的重要性。

Highlights

Ilya Sutskever 是 OpenAI 的联合创始人兼首席科学家，致力于构建人工通用智能（AGI）以造福全人类。

Ilya 在俄罗斯出生，五岁时移居以色列，并在多伦多大学完成数学学士学位。

Ilya 认为人工神经网络与生物神经网络在某种程度上是相似的，这种假设推动了深度学习的发展。

Ilya 强调了反向传播算法的重要性，这是一种让人工神经网络通过经验学习的数学方程。

大型语言模型（如GPT-3）通过训练神经网络猜测文本中的下一个单词，从而提高其预测能力。

Ilya 认为，尽管人工智能在某些方面超越了人类，但在学习方式上，人类和机器仍然存在显著差异。

OpenAI 的使命是确保人工通用智能（AGI）造福全人类，从非营利和开源开始，现已转变为有利润和闭源的公司。

Ilya 讨论了 OpenAI 与微软的合作关系，以及如何确保投资者的财务责任与 OpenAI 使命的一致性。

Ilya 认为，随着 AI 能力的增长，未来可能会有一天出于安全考虑而不再开源这些模型。

Ilya 讨论了意识问题，以及 AI 是否可能发展出意识，他认为这是一个复杂且难以定义的概念。

Ilya 认为深度学习将继续取得进展，不仅仅是通过扩大模型规模，还包括在多个层面上改进深度学习堆栈。

Ilya 强调了在 AI 研究中，信任自己的直觉和独特见解的重要性。

OpenAI 的成功关键绩效指标（KPI）包括技术进步、研究执行情况以及对 AI 系统的控制和引导。

Ilya 认为，尽管 AI 技术已经整合到团队工作中，提高了生产力，但目前对团队动态的影响并不显著。

Ilya 讨论了 OpenAI 企业结构的独特性，它是一种有利润上限的公司，最终将转变为非营利组织。

Ilya 认为，未来的 AI 系统将在多个小的改进和一些大的改进的基础上，变得更加强大和复杂。

Ilya 讨论了在 AI 领域中，通用训练与专门训练的优劣，以及如何结合两者以取得最佳效果。

Ilya 认为，尽管 AI 模型如 GPT-3 在诗歌创作方面表现出色，但在团队工作中，它们目前还没有显著改变团队动态。

Transcripts

who you are defines how you build

welcome YouTube and Stanford communities

to the entrepreneurial thought leaders

seminar

um brought to you by stvp the

entrepreneurship Center in the School of

Engineering at Stanford and basis The

Business Association of Stanford

entrepreneurial students today we are so

honored to have Ilia suitskiver here at

ETL Ilya is the co-founder and chief

scientist of open AI which aims to build

artificial general intelligence for the

benefit of all Humanity Elon Musk and

others have cited that Ilya is the

foundational mind behind the large

language model generative pre-trained

Transformer 3 or gpt3 and its

public-facing product chat gbt a few

product releases have created as much

excitement Intrigue and fear as the

release of chat gbt in November of 2022.

Ilia was Ilia is another example of how

the U.S and the world has been the

beneficiary of amazing talent from

Israel and Russia is Elia was born in

Russia and then when he was five he

moved to Israel where he grew up and he

spent um at the first half of undergrad

even in Israel and then he transferred

and went to the University of Toronto to

complete his bachelor's degree in

mathematics he went on to get a master's

in PhD in computer science from the

University of Toronto and then came over

here to the farm and did a short stint

with Andrew ing before returning back

to Toronto to work under his advisor

Jeffrey Hintz research company DNN

research Google then acquired DNN

research shortly thereafter in 2013 and

Ilya became a research scientist as part

of Google brain and in 2015 he left

Google to become a director of the then

newly formed open AI it's hard to

overestimate the impact that chat gbt

has had on the world since its release

in November of last year and while it

feels like chat gbt came out of nowhere

to turn the world on its head the truth

is there's a deep history of innovation

that has led to that moment and as

profound as chat gbt is Ilia is No

Stranger in uttering in discontinuous

leaps of innovation and AI Jeff Hinton

has said that Ilya was the main impetus

for Alex net which was the convolutional

neural network in 2012 that is

attributed to setting off the deep

learning Revolution that has led to the

moment that we are now in and of course

it was seven years since the founding of

open AI that chat GB T was finally

Unleashed to the world Ilyas was elected

a fellow of the Royal Society in 2022

he's been named to the MIT tech review

35 under 35 list in 2015. he's received

the University of Toronto's innovator of

the Year award in 2014 and the Google

graduate Fellowship from 2010 to 2012.

so with that everybody please give a

virtual warm Round of Applause and

welcome for Ilia to the entrepreneurial

thought leader seminar so Ilya imagine

lots of Applause and you're always

invited back onto the farm physically

whenever you are able so Ilya there's so

much to discuss and I know we're gonna

have solo time and we have quite a broad

range of fluency around the audience in

terms of chat gbt and lot large language

models I wanted to start off with just a

quick question on the technology which

is just the key technology underlying

open Ai and generative AI more broadly

is large language models can you

describe the technology in simple terms

and now that you're at the Forefront of

the tech can you share would have

surprised you the most about what the

tech can do that you didn't anticipate

yeah

I I can't explain

well this technology is and why it works

I think the explanation for why it works

is both simple and extremely beautiful

and it works for the following reason

so you know how

the human brain is our best example of

intelligence

in in the world

and we know that the human brain is made

out of a large number of neurons a very

very large number of neurons

neuroscientists have studied neurons for

many decades to try to understand how

they work precisely and while the

operation of our biological neurons are

still mysterious

there's been a pretty bold conjecture

made by the earliest deep learning

researchers in the 40s

the idea that an artificial neuron

the ones that we have in our artificial

neural networks kind of sort of similar

to a biological neuron if you squint so

that's there's an assumption there

and we can just run with this assumption

now one of the nice things about these

artificial neurons is that you can

they are much simpler and you can study

them mathematically

and a very important breakthrough that

was done by the very very early deep

learning Pioneers before it was known as

deep learning was the discovery of the

back propagation algorithm

which is a mathematical equation for how

these artificial neural networks should

learn

it

provides us with a way of

taking a large computer

and implementing this neural network in

code and then there would be there is an

equation that we can code up that tells

us how this neural network should adapt

its connections to learn from experience

now a lot of additional further progress

had to do with understanding just how

good and how capable this learning

procedure is

and what are the exact conditions under

which this learning procedure works well

it's although this is although we do

with computers it was a little bit of an

experimental science a little bit like

biology where you have something that is

you know like like like a local

biological experiment a little bit

and

so then a lot of the progress with deep

learning basically boils down to this

we can build these neural networks in

our large computers and we can train

them on some data we can train those

large neural networks to do whatever it

is that the data asks them to do

now the idea of a large language model

is that if you have a very large neural

network

perhaps one that's now not that far from

like these neural networks are pretty

large

and we train them on the task

to guess the next word from a bunch of

previous words

in text

so this is the idea of a large language

model you train a big neural network to

guess the next word from a previous from

the previous words in text and you want

the neural network to guess the next

word as accurately as possible

now the thing that happens here is we

need to come back to our original

assumption that maybe biological neurons

aren't that different from artificial

neurons and so if you have a large

neural network like this that guesses

the next word really well maybe it will

be not that different from what people

do when they speak and that's what you

get so now when you talk to

a neural network like this it's because

it has such a great such an excellent

sense of what comes next what word comes

next it can narrow down it can't see the

future but it can narrow down the

possibilities correctly from its

understanding

being able to guess what comes next very

very accurately requires prediction

which is the way you operationalize

understanding

what does it mean for a neural network

to understand it's hard to come up with

a clean answer but it is very easy to

measure and optimize the Network's

prediction error of the next word

so we say we want understanding but we

can optimize prediction and that's what

we do

and that's how you get this current

large language models these are neural

networks which are large they are

trained with the back propagation

algorithm which is very capable and if

you allow yourself to imagine that an

artificial neuron is not that different

from a biological neuron then yeah like

our brains are doing are capable of

doing a pretty good job at guessing the

next word if you pay if you pay very

close attention

so so if I let I love that and I just

want to make this more concrete so just

to push that analogy further between the

biological brain and these neural um uh

analog physical networks digital

networks

um if the human if if we consider you

know before it was considered untenable

for these machines to learn now it's a

given that they can learn or do this

um uh do predictive outcomes of what's

going to come next if a human is at 1X

learning and you have the visibility

into the most recent chat gbt models

what would you put the most recent chat

gbt model as a ratio of where the humans

are at so if humans are at 1X what's

chat gpdn

you know it's a bit hard to make direct

comparisons between our artificial

neural networks and people because at

present

people

are able

to learn more from a lot less data

this is why these neural networks like

Chad GPT are trained on so much data to

compensate for their initial slow

learning ability

you know as we train these neural

networks and we make them better

faster learning abilities start to

emerge

but overall

overall it is the case that we are we

are quite different the way people learn

is quite different from the way these

neural networks learn like one example

might be you know these neural networks

they are

you know solidly good at math or

programming

but like the amount of math books they

needed to get let's say good at

something like

calculus is very high or as a person

would need a fairly you know two

textbooks and maybe 200 exercises and

you're pretty pretty much good to go

so there is just to get an order of

magnitude sense if you relax the data

constraint so if you let the machine

consume as much data as it needs do you

think it's operating at like one-tenth

of a human right now or

you know it's quite hard to answer that

question still and let me tell you why I

hesitate to like I think that any figure

like this will be misleading and I want

to explain why like because right now

any such neural network is obviously

very superhuman when it comes to the

breadth of its knowledge and to the very

large number of skills that these neural

networks have for example they're very

good at poetry and they're very good you

know like they can talk eloquently about

any topic pretty much

and they can talk about historical

events and lots of things like this

on the other hand on the other hand

people

can go deep and they do go deep so you

may have an expert like someone who

understands something very deeply

despite having read only a small amount

of documents let's say on the topic so

because of this difference I really

hesitated to answer the question in

terms of oh yeah it's like some some

number between zero do you think there

is a singularity point where the

machines will surpass the humans in

terms of the pace of learning and

adaption yeah and when do you think that

point will occur

I don't know I don't know when it will

occur I think some additional advances

will need to do will happen but

you know I absolutely would not bet

against this point occurring at some at

some point can you give me a range is it

at some point next month is it next year

you know

I think it's like the the uncertainty on

this thing is quite High because these

advances I can imagine it can take in

quite a while I can imagine it can take

any disappoint in a long time I can also

imagine it's taking

you know

some

number of years but it's just very it's

very hard to give a Cali braided answer

and I I know there's lots of push

forward so I'm going to ask one more

question then move on to some of the

other issues but um I know I read that

when you were a child you were disturbed

by the notion of Consciousness and I

wasn't sure what that what that word

meant Disturbed but I'm curious do you

view Consciousness or sentience or

self-awareness as an extenuation of

learning do you think that that is

something that also is an inevitability

that will happen or not

yeah I mean on the Consciousness

questions

like yeah I was as a child that would

like you know look into my in my hand

and I would be like how can it be that

this is my hand that I get to see like I

something of this nature I don't know

how to explain it much better so that's

been something I was curious about

you know it's

It's Tricky with Consciousness because

how do you define it it's something that

the looted definition for a long time

and how can you test it in a system

maybe there is a system which acts

perfectly right but um perfectly the way

you'd expect

um

a conscious system would act yet maybe

it won't be conscious for some reason I

do think there is a very simple way to

there's there is an experiment which we

could run on an AI system which we can't

run on which we can't run just yet but

maybe

in like the Future Point when the AI

learns very very quickly from less from

less data we could do the following

experiment

very carefully with very carefully

curate the data

such that we never ever mention anything

about consciousness it would only say

you know here is

here's a ball and here's a castle and

here is like a little toy like you would

imagine imagine you'd have data of this

sort it would be very controlled maybe

we'd have some number of years worth of

this kind of training data

maybe it would be maybe such an AI

system would be interacting with a lot

of different teachers learning from them

but all very carefully you never ever

mentioned Consciousness you don't talk

about

people don't talk about

anything except for the most surface

level Notions of their experience and

then at some point you sit down this Ai

and you say Okay I want to tell you

about Consciousness it's the stain

that's a little bit not well understood

people disagree about it but that's how

they describe it and imagine if the AI

then goes and says oh my god I've been

feeling the same thing but I didn't know

how to articulate it that would be okay

that would be definitely something to

think about it's like if the AI was just

trained on very mundane data around

objects and going from place to place or

maybe you know

something like this from a very narrow

set of Concepts we would never ever

mention that and if it could somehow

eloquently correctly talk about it in a

way that we would recognize that would

be convincing

and do you think of it as a some as

Consciousness as something of degree or

is it something more binary uh

I think it's something that's more a

matter of degree

I think that

I think that like you know

let's say if a person is very tired

extremely tired and maybe drunk then

perhaps if that's when when someone is

in that state and maybe their

Consciousness is already reduced to some

degree

I can imagine that animals have a more

reduced form of Consciousness if you

imagine going from

you know large primates maybe dogs cats

and then eventually you get mice you

might get an insect like

feels like I would say it's pretty

continuous yeah

okay I want to move on even though I

could I would love to keep asking more

questions along the lines of the

technology but I want to move on to

talking about the mission of openai and

how you perceive or any issues around

ethics and your role as Chief science

officer how ethics informs if at all how

you think about your role and so let me

just lay a couple Foundation points out

and then have you speak

um as you know open ai's mission is to

ensure the art of that artificial

general intelligence benefits all of

humanity and it started off as a

non-profit and open source and it is now

a for-profit and closed-sourced and with

a close relationship with Microsoft and

Elon Musk who I believe recruited you to

originally join open Ai and gave 100

million dollars when it was a non-profit

has says that the original Vision was to

create a counterweight to Google and the

corporate world and he didn't want to

have a world in which AI which is has

which he perceives and others can have

an existential threat to humanity to be

solely in the holds of of corporate of a

for-profit

um and now

open AI is neither open nor exclusively

a non-profit it's also a for-profit with

close ties to Microsoft and it looks

like the world may be headed towards

um a private duopoly between Microsoft

and Google

can you shed light on the calculus to

shift from a for-profit to a non-profit

and did you weigh in the ethics of that

decision and do ethics play a role in

how you conceive of your role as the

chief science officer or do you view it

more as something that somebody else

should handle and you are mainly just

tasked with pushing the technology

forward

yeah so this question is many parts let

me yeah let me think about the best way

to to approach it

so there are several parts there is the

there is the question around open source

versus closed source

there is a question around non-profit

versus for-profit

and the connection with Microsoft

and how to see that in the context of

Elon musk's recent comments

and then the question about how I see my

role in this maybe I'll start with that

because I think that's easier Okay so

I feel

yeah

the way I see my role

I feel a lot I I feel

direct responsibility for whatever open

AI does even though

I my role is primarily around advancing

the science

it is still the case I'm one of the

founders of the company

and

ultimately I care a lot

about open ai's overall impact

now I want to go so with this context I

want to go and talk about the open

source versus closed source

and the non-profit versus for-profit

and I want to start with open source

which is closed source

because I think that

you know the challenge with AI

is that AI is so all encompassing

encompassing

and it comes with many different

challenges it comes with many many

different dentures

which come into conflict with each other

and I think the open source versus

closed source is a great example of that

why is it desirable well let me put it

this way

what are some reasons

for which it is desirable to open source

AI

the answer there would be

to

prevent

concentration of power

in the hands of those who are building

the AI so if you are in a world where

let's say there is only a small number

of companies you might that control this

very powerful technology

you might say this is an undesirable

world and that AI should be open and

that anyone could use the AI this is the

argument for open source

but this argument

you know of course you know to State the

obvious there are near-term commercial

incentives against open source but there

is another longer term argument against

open sourcing as well which is

if we believe

if one believes that eventually AI is

going to be unbelievably powerful

if we get to a point where your AI is so

powerful where you can just tell it hey

can you autonomously

create a biological I don't know a

biological research lab

autonomously

do all the paperwork render space hire

the technicians aggregate experiments do

all this autonomously

like that starts to get

incredible that starts to get like

mind-bandingly powerful should this be

open sourced also

so my position on the open source

question is that I think that

I think that there is a maybe

a level of capability you can think

about these neural networks in terms of

capability

how capable they are how smart they are

how much how many how much how much can

they do

when the capability is on the lower end

I think open sourcing is a great thing

but at some point and you know there can

be debate about where the pointer is

but I would say that at some point the

capability will become so vast that it

will be obviously irresponsible to open

source models and was that the driver

Behind Closed sourcing it or was it

driven by a a devil's compact or

business necessity to get cash in uh

from Microsoft or others to support the

viability of the business was the

decision making to close it down

actually driven by that line of

reasoning or was it driven by more so

it's so so the way I'd articulate it you

know my view is that the current level

of capability is still not that high

where it will be the safety

consideration it will drive the close

closed Source in the model this kind of

this kind of research

so in other words a claim that it goes

in phases right now it is indeed the

competitive phase but I claim that as

the capabilities of these models keep

increasing

there will come a day where it will be

the safety consideration that will be

the obvious and immediate driver to not

open source these models

so this is the open source versus closed

Source but your question had enough but

your question in another part which is

non-profit versus for-profit

and we can talk about that also

you know indeed it would be preferable

in a certain meaningful sense if open AI

could just be a for a non-profit from

now until the mission of open AI is

complete

however one of the things that's worth

pointing out is the very significant

cost of these data centers I'm sure

you're reading about various AI startups

and the amount of money they are raising

the great majority of which goes to the

cloud providers

why is that

well the reason so much money is needed

is because

this is

the nature of these large neural

networks they need the compute end of

story you can see something like this

that's all you can see a divide that's

now happening between Academia

and the AI companies

so for a long time for many decades

Cutting Edge research in AI took place

in academic departments in universities

that cap being the case up until the

mid-2010s

but at some point when the complexity

and the cost of this project started to

get very large it no longer remained

possible for universities to be

competitive and now universities need a

University Research in AI needs to find

some other way in which to contribute

those ways exist they're just different

from the way they're used to and

different from the way the companies are

contributing right now

now

with this context you're saying okay

the thing about non-profit a non-profit

is the people who give money to a

non-profit never get to see any any of

it back it is a real donation

and believe it or not it is quite a bit

harder to convince people to give money

to a non-profit and so we so so we think

what's what's the solution there or what

is a good course of action

so we came up with an idea

that to my knowledge is unique

in all corporate structures in the world

the open air corporate structure is

absolutely unique

open AI is not a for-profit company it

is a capped profit company

and I'd like to explain what that means

what that means

is that

equity in open AI can be better seen as

Bond rather than equity in a normal

company the main feature of a bond is

that once it's paid out it's gone

so in other words

open AI has a finite obligation to its

investors as opposed to an infinite

obligation to that normal companies have

and does that include the founders do

the founders have equity in open AI

so Sam Altman does not have equity

but the other Founders do and is it

capped or is it unlimited it is capped

and how does that cap is that capped at

because the the founders I presume

didn't buy in unless it's capped at the

nominal

Share value

um

I'm not sure I understand the question

precisely but what I can say like what

what I can answer the part which I do

understand which is like

there is certainly like it isn't there

are it is a different it is different

from normal startup Equity but there are

some similarities as well where the

earlier you join the company

the higher the cap is because then

the larger cap is needed to attract the

initial investors as the company

continues to succeed the cap decreases

and why is that important it's important

because it means that the company

one once when once all the obligation to

investors and employees are paid out

open AI becomes a non-profit again

and you can say this is totally crazy

what are you talking about like it's not

going to change anything

but it's worth considering what we

expect like it's worth

looking at what we think AI will be

I mean we can look at what AI is today

and I think it is not at all

inconceivable

for open AI tool achieve its

to pay out its obligation to the

investors and employees become a

non-profit at around the time when

perhaps the computers will become so

capable where the economic destruction

will be very big where this transition

will be very beneficial

so this is the answer on the cap profit

versus

non-profit

there was a last part to your question I

know I'm speaking for a while but the

question had many parts the last part of

your question is the Microsoft

relationship

and

so here

the thing that's very fortunate

is that Microsoft is a

there thinking about these questions the

right way they understand

the potential and the gravity of AGI and

so for example on the on all the

investor documents that any investor in

open AI has signed and by the way

Microsoft is an investor into open AI

which is a very different relationship

from the

deepmind

any anyone who signed any document any

investment document there is a a purple

rectangle at the top of the investment

document which says that the fiduciary

duty of open AI is to the open AI

mission

which means that you run the risk of

potentially losing all your money if the

mission comes in conflict

so this is something that all the

investors have signed

and let me just make this clear for

everybody because Google Google acquired

deepmind so deepmind was just an asset

inside of Google but beholden to Google

you're making the distinction that with

openai Microsoft is an investor and so

beholden to this fiduciary duty for the

mission of openai which is held by the

non-profit which is a is is a a GP or an

LP in the

um in in the for-profit

um okay understood yeah so it's not

something like this you know I am

you know there are people

I can't tell you the precise details

yeah but so but this is the general

picture

and you know some have claimed though

now especially it uh um Steve Wozniak

the co-founder of apple and Elon Muska

famously signed this very public

petition saying that the point of no

return is already passed or we're

approaching it where it's going to be

impossible to reign in Ai and it's and

it's it's repercussions if we don't halt

it now and they've called for halting AI

um

I'm curious

on you are a world citizen Ilia you were

born in Russia you were raised in Israel

you're Canadian

um and I'm and it's open ai's response

to that public petition was

um I know Sam basically said that you

know this wasn't the right way to go

about doing that but also in parallel

Sam is on a world tour with many

countries that also can be antagonistic

towards the West

are there any citizen obligations

ethical obligations that you think also

overweigh your your technological

obligations when it comes to spreading

the technology around the world right

now through open AI do you think that

should be beholden to a regulation or

some oversight

let me think

once again the question had a number of

Parts did I apologize I'm trying to give

you the so you can respond however you

want to on that I know we're going to

come out of off of time so I just want

to give you the mic and just share

everything that's on my mind and you can

decide how you want to handle it yeah

thank you

I mean

you know

it is true

that AI is going to become truly

extremely powerful and truly extremely

transformative

and I do think

that we will want to move to a world

with sensible government regulations and

there you know there are several

Dimensions to it

we want to be in a world where there are

clear rules about for example training

more powerful neural networks

we want there to be some kind of careful

evaluation careful prediction of these

of what we expect these neural networks

of what they can do today and on what we

expect them to be able to do let's say

in a year from now or by the time they

finish training I think all these things

will be very necessary in order to

like rational like rationally

I wouldn't use the word slow down the

progress I would use the term you want

to make it so that the progress is

sensible so that with each step we've

done the homework and indeed we can make

a credible story that okay

the neural network the system that we've

trained it has we are doing this and

here all the steps and it's been

verified or certified I think that is

the world that we are headed to which I

think is correct

and as for the citizen obligation I feel

like

I mean

15 what

I'll answer it like this

like I think I think like there are

there are two answers to it so obviously

you know I live I live in the United

States and I really like it here and I

want and I want this place to flourish

as much as possible

I care about that

I think that of course there will be

lots of but the world is much more than

just the US

and I think that these are the kind of

questions which I feel a little bit

let's say outside of my expertise how

these

between country relationships work out

but I'm sure there will be lots of

discussions there as well

yeah

um Julia can I turn a little bit towards

strategy

um I'm curious for you guys internally

what metrics do you track as your North

Star what are the most sacred kpis that

you use to measure open ai's success

right now

the most sacred kpis

you know I think this is also the kind

of question where maybe different people

will give you different answers

but I would say I would say that there

are

if I were to really narrow it down

I would say that there are

there is a couple of really important

kpi of a really important dimensions of

progress

one is undeniably the technical progress

are we doing good research

do we understand our systems better are

we able to train them better can we

control them better I is our

is ours is our research plan being

executed well is our safety plan being

executed well how happy are we with it I

would say this would be my description

of the primary kpi which is do a good

job of the technology then there is of

course stuff around the product but

which I think is cool but I would say

that it is really the core technology

which is the heart of openai the

technology its development

and on end

its control

it's steering

and and do you view um right now chat

gbt is a destination

do you view open AI in the future being

a destination that people go to like

Google or will it be powering other

applications and be the back end or be

be you know used as part of the back end

infrastructure

um is it a destination or is it going to

be more behind the scenes

um in in five to ten years

yeah well I mean things change so fast I

I cannot make any claims about

five to ten years in terms of the

correct shape of the product I imagine a

little bit of both perhaps but this kind

of question

I mean I think it remains to be seen I

think there are I think this stuff is

still so new

okay

I'm gonna ask one more question I'm

gonna jump to the student questions if

you're a student at Stanford today

interested in AI if you were you know

somebody who wants to be Ilia um what

would you focus your time and another

second question on this if you're also

interested in entrepreneurship

um where would you what would you what

advice would you give for a Stanford

undergrad engineer that's interested in

Ai and Entrepreneurship

so

I think on the first one

it's always hard to give generic advice

like this

but

I can still provide some generic advice

nonetheless

and I think it's something like

it it is generally a good idea to lean

into

one's unique predispositions

you know every you know why if you think

if you look if you think about the set

of let's say inclinations or skills or

talents that the person might have the

combination is pretty rare so leaning

into that is a very good idea no matter

which direction you choose to go look to

going and then on the AI research

like I would say

I would say that there

you know

I could say something but even but there

especially you want to lean into your

own ideas and really ask yourself what

can you is is there something

that's totally obvious to you that makes

you go why is everyone else not getting

it if you feel like this that's a good

sign it means that you might be able

that that you you want to lean into that

and explore it and see if your instinct

is true or not it may not be true

but you know my my advisor Jeff Hinton

says this thing which I really like he

says you should trust your intuition

because if your intuition is good you go

really far and if it's not good then

there's nothing you can do

hmm

and as far as entrepreneurship is

concerned

I feel like

this is a place where the unique

perspective is even more valuable or

maybe equally it's because it's maybe

maybe I'll I'll explain why I think it's

more valuable than in research well in

research it's very valuable too but in

entrepreneurship like you need to like

almost pull from your unique life

experience where you say okay I see this

thing I see this technology I see

something like take a very very Broad

View and see if you can hone in on

something and then actually just go for

it

so that would that would be the

conclusion of my generic advice okay

which is great that's also great I'm

going to move on to the student question

so one of the most upvoted question is

how do you see the field of deep

learning evolving in the next five to

ten years

let's see

you know I expect deep learning to

continue to make progress I

I expect that

you know there was a period of time

where

a lot of progress came from scaling

and you you saw that most in the most

pronounced way in going from GPT 1 to

gpd3

but

things will change a little bit the

reason the reason that the reason that

progress in scaling was so rapid is

because people had all these data

centers which they weren't using for a

single training run

so by simply reallocating existing

resources you could make a lot of

progress

and it doesn't take that long

necessarily to reallocate existing

resources you just need to you know

someone just needs to decide to do so

it is different now because the training

runs are very big and the scaling is not

going to be progressing as fast as it

used to be because building data center

takes time

but at the same time I expect deep

learning to continue to make progress in

uh from other places the Deep learning

stack is quite deep and I expect that

there will be improvements in many

layers of the stack and together they

will still lead to progress being very

robust

and so

if I had to guess I'd imagine that there

would be maybe

I'm certain we will discover new

properties which are currently unknown

of deep learning and those properties

will be utilized and I fully expect that

the systems of five to ten years from

now will be much much better than once

they are we have right now

but exactly how it's going to look like

I think

I think it's a bit harder to answer it's

a bit like

it's because the improvements that there

is there will be maybe a small number of

big improvements and also a large number

of small improvements all integrated

into a large complex engineering

artifact

and can I ask your you know your

co-founder Sam Altman has said that

we've reached the limits of what we can

achieve by scaling to larger language

models is do you agree

um and if so you know what then what is

the next Innovation Frontier that you're

focusing on if that's the case yeah so

I think maybe

I don't remember I don't know exactly

what he said but maybe he meant

something like that the age of easy

scaling has ended or something like this

like of course of course the larger

neural Nets will be better but it will

be a lot of effort and cost to do them

but I think there will be lots of

different Frontiers and actually into

the question of how can one contribute

in deep learning identifying such a

frontier perhaps one that's been missed

by others is very fruitful

and is it can I go even just deeper on

that because I think there is this

debate about vertical Focus versus

General

um uh General's training you know is it

better do you think there's better

performance that can be achieved in

particular domains such as law or

Medicine by training with special data

sets or is it likely that generalist

training with all available data will be

more beneficial

so

like at some point we should absolutely

expect Specialists training to make a

huge impact

but the reason we do the generalist

training

is just so that we can even reach the

point where

just so that we can reach the point

where the neural network can even

understand the questions that we are

asking and only when it has a very

robust understanding only then we can go

into specialist training and really

benefit from it so yeah I mean I think

all these I think these are all fruitful

directions

but you don't think when do you think

we'll be at that point when specialist

training

is the thing to focus on I mean

you know like if you look at people who

do open source work people who work with

open source models they do a fair bit of

this kind of specialist training because

they have a fairly underpowered model

and they try to get any ounce of

performance they can out of it

so I would say that this is an example

I'd say that this is an example of it

happening like it's already happening to

some degree it's not a binary it's you

might want to think of it as of like a

continual Spectrum but do you think that

the competitor do you think that the

winning Advantage is going to be having

these proprietary data sets or is it

going to be having a much higher

performance large language model when it

comes to these applications of AI into

verticals

so I think it may be productive to think

about about an AI like this as a

combination of multiple factors where

each factor makes a contribution

and

is it better to have a special data

which helps you make

your AI better in a particular set of

tasks of course is it better to have a

more capable base model of course from

the perspective of the task so maybe

this is the the answer it's not an

either or

I'm going to move down to the other

questions

um there's a question on what was the

cost of training and developing GPT T3

slash four

yeah so

you know for for obvious reasons I can't

comment on that

um but there I think there is a you know

I think even from our research Community

there's a strong desire to be able to

get access to

um uh to different aspects of open ai's

technology and are there any plans for

releasing it to researchers or to other

startups to encourage more competition

and Innovation some of the requests that

I've heard are unfettered interactions

without safeguards to understand the

model's performance model specifications

including details on how it was trained

and access to the model itself I.E the

trained parameters do you want to

comment on any of that

yeah I mean I think

like it's related to our earlier

question about open versus closed

I think that there are some

intermediate approaches which can be

very fruitful

for example

model access and various combinations of

that

can be very very productive because

these mineral networks already have such

a large and complicated surface area of

behavior

and

and studying that alone can be extremely

interesting look if you have an academic

access problem we provide various forms

of access to the models and in fact

plenty of academic research Labs do

study them in this way

so

I think this kind of approach

is viable

and it's something that we could that we

are doing

I know we're coming up on time I want to

end with just one final question which

is can you just share any unintuitive

but compelling use cases for how you

love to use chat gbt that others may not

know about

um

so I mean I don't I wouldn't say that

it's unknown but I I really enjoy its

poem writing ability

it can write poems it can rap it can it

can be it can be it can be pretty

amusing

and do you guys use it is it is it an

integrated part of the

um of teamwork at open I assume it is

but I'm curious do you have any insights

on how it changes Dynamics with teams

when you have ai deeply integrated into

you know a human team and how they're

working and any insights into to what we

may not know but that will come

I would say I would say to today the

best way to describe the impact is that

everyone is a little bit more productive

people are a little bit more on top of

things I wouldn't say that right now

there is a dramatic impact on Dynamics

which I can say oh yeah the Dynamics

have shifted in this pronounced way

okay I'm curious if it depersonalizes

conversations because it's the AI bot or

maybe it but maybe we're not at that

point yet where it's specifically

that I definitely I I don't think that's

the case and

I predict that will not be the case but

we'll see

well thank you Ilya for a fascinating

discussion time is always too short

you're always invited back to the farm

um we'd love to have you either

virtually or in person

um so thank you thank you thank you um

to our audience thank you for tuning in

for this session of the entrepreneurial

thought leader series next week where

we're going to be joined by the

executive chairman and co-founder of

OCTA Frederick karist and you can find

that event and other future events in

this ETL series on our Stanford e-corner

YouTube channel and you'll find even

more of the videos podcasts and articles

about entrepreneurship and Innovation at

Stanford e-corner that's

ecorner.stanford.edu and as always thank

you for tuning in to ETL

thank you