Geoffrey Hinton is a genius | Jay McClelland and Lex Fridman

Summary

视频脚本讲述了科学家杰夫·辛顿（Jeff Hinton）在神经网络和深度学习领域的贡献。辛顿与同事一起创建了PDP研究小组，推动了交互激活模型的研究。他们意识到，为了优化神经网络的学习，应该从生物学启发转向定义问题并调整连接权重来解决问题。这导致了梯度下降和反向传播算法的发展，后者通过从输出层向隐藏层反向传播错误信号来调整权重。辛顿的直觉式思考和对深度学习工作原理的直观解释，以及他对计算智能和认知机制的深刻理解，对视频讲述者产生了深远影响。视频还探讨了人类智能与计算智能的关系，以及如何通过构建超越生物限制的计算智能来扩展人类智能的可能性。

Takeaways

• 👨‍🔬 杰夫·辛顿（Jeff Hinton）是深度学习和神经网络领域的先驱，他对于反向传播算法（Backpropagation）的发展做出了重要贡献。
• 🔄 反向传播算法的核心思想是定义一个目标函数，并通过梯度下降（Gradient Descent）来调整连接权重，以最小化错误或损失。
• 📈 梯度下降的概念在工程领域已经为人所熟知，并且有一个称为delta规则的算法，由斯坦福大学的Woodrow和Hoff发明。
• 🧠 保罗·沃尔姆哈特（Paul Werbos）推广了delta规则，使其能够应用于多层神经网络，从而形成了广义delta规则，也就是后来的反向传播算法。
• 🤔 辛顿当时对反向传播算法的效果持怀疑态度，并提出了基于balsa机器概念的替代算法，但最终反向传播算法被证明更为有效。
• 🌟 辛顿在1981年发表了两篇开创性的论文，一篇关于transformers的概念，另一篇关于语义认知的早期研究，对认知科学产生了深远影响。
• 📚 辛顿在1977年未发表的一篇论文中描述了神经网络如何进行递归计算，这一想法他持续探索多年。
• 💡 辛顿的思考方式对理解人类数学认知产生了深远影响，他倾向于用直观的方式解释深度学习的工作原理，而不是使用复杂的数学方程。
• 🧵 辛顿的工作与布尔逻辑和理论物理相联系，特别是他在80年代中期关于Boltzmann机的研究，将布尔传统带入了更连续的概率和分级约束满足领域。
• 🧬 辛顿希望对推理本身做出贡献，不仅仅是人类的推理，而是更广泛地理解推理的机制。
• 🧠 人类智能只是智能的一种表现形式，辛顿和其他研究者正在探索的计算智能领域可能超越人类自身的生物学限制。
• 🚀 深度学习在谷歌大脑、OpenAI和DeepMind等项目中展现出巨大潜力，通过大规模计算解决了诸如围棋和蛋白质折叠等问题。

Q & A

杰弗里·辛顿（Jeff Hinton）在神经网络领域有哪些贡献？

杰弗里·辛顿是深度学习和神经网络领域的先驱之一。他提出了反向传播（Backpropagation）算法，这是一种用于训练神经网络的监督学习技术。此外，他还对玻尔兹曼机（Boltzmann Machine）进行了研究，这是一种随机递归神经网络，能够学习数据的概率分布。辛顿的工作对理解人类认知和推理有深远的影响。

什么是反向传播算法，它在神经网络中的作用是什么？

反向传播算法是一种用于训练多层神经网络的监督学习技术。它通过计算损失函数关于网络参数的梯度，并利用这些梯度来更新网络的权重。反向传播包括两个过程：正向传播和误差反向传播。在正向传播中，输入数据通过网络传播以产生输出；在误差反向传播中，计算输出和目标值之间的误差，并将这个误差信号反向传播回网络，以调整权重，减少误差。

玻尔兹曼机是如何与布尔逻辑和概率理论相结合的？

玻尔兹曼机是一种随机神经网络，它结合了布尔逻辑和概率理论。布尔逻辑是关于推理和命题逻辑的数学理论，而概率理论处理不确定性和随机性。在玻尔兹曼机中，神经元的状态是随机确定的，反映了概率性，而网络的学习过程则涉及到逻辑和推理，因为它试图找到数据的概率分布。

杰弗里·辛顿在1981年的两篇论文中提出了哪些前瞻性的想法？

在1981年，杰弗里·辛顿发表了两篇具有前瞻性的论文。一篇是关于变换器（transformers）的概念，这是一种在自然语言处理中非常重要的模型，用于处理序列数据。另一篇是关于语义认知的早期论文，这篇论文启发了对认知的语义方面的研究。

在深度学习中，为什么需要定义一个目标函数？

在深度学习中，定义一个目标函数是为了量化模型的性能，即模型输出与实际值之间的差异。这个函数通常被称为损失函数或误差函数。通过最小化目标函数，可以调整网络的权重，使得模型的预测尽可能接近于真实数据，从而提高模型的准确性。

梯度下降法在神经网络训练中的作用是什么？

梯度下降法是一种优化算法，用于最小化神经网络的损失函数。通过计算损失函数相对于网络权重的梯度，梯度下降法逐步调整权重，以减少预测误差。这是训练神经网络中的一个关键步骤，因为它指导了网络如何通过学习数据来改进其性能。

杰弗里·辛顿的研究对理解人类认知有哪些影响？

杰弗里·辛顿的研究对理解人类认知有重要影响。他的工作不仅推动了神经网络和深度学习领域的发展，而且提供了关于大脑如何处理信息、如何进行推理和决策的新见解。特别是，他对玻尔兹曼机的研究，将逻辑和概率理论结合起来，为理解人类大脑的工作方式提供了新的视角。

为什么说反向传播算法是深度学习中的关键技术？

反向传播算法是深度学习中的关键技术，因为它允许对多层神经网络进行有效的训练。在多层网络中，直接计算从输入到输出所有层的权重变化是非常复杂的。反向传播算法通过利用链式法则，从输出层开始，逐层反向计算误差梯度，从而简化了这一过程，使得训练复杂的深层网络成为可能。

在深度学习中，为什么需要考虑计算的规模？

在深度学习中，计算的规模至关重要，因为模型的性能往往随着模型大小和训练数据量的增加而提高。较大的模型可以捕捉更复杂的特征，而更多的数据可以提供更丰富的信息，帮助模型更好地泛化。此外，大规模计算还可以利用并行处理和分布式计算的优势，加快训练过程。

杰弗里·辛顿的哪些思维方式对您产生了影响？

杰弗里·辛顿的直观和视觉化的思维方式对提问者产生了深远的影响。辛顿倾向于通过图形和几何隐喻来解释深度学习的概念，而不是仅仅依赖于数学方程。这种直观的解释方式有助于更深刻地理解复杂的概念，并且使得这些概念更加易于理解和记忆。

如何理解杰弗里·辛顿在1977年未发表的论文中描述的神经网络进行递归计算的想法？

在1977年的未发表的论文中，杰弗里·辛顿提出了一个关于神经网络进行递归计算的想法。这个想法涉及到在调用子程序时保存当前状态，并在子程序执行完毕后恢复这一状态，以便继续之前的计算。辛顿提出，可以通过快速调整连接权重来保存和恢复状态，这种方法后来成为了深度学习中处理序列数据的重要概念。

为什么说深度学习的发展与跨学科合作有关？

深度学习的发展与跨学科合作密切相关，因为这个领域结合了计算机科学、神经科学、认知科学、数学和物理学等多个学科的理论和方法。跨学科合作促进了不同领域知识的融合，推动了深度学习算法和模型的创新。例如，玻尔兹曼机的提出就是逻辑、概率理论和物理学结合的产物。

Outlines

00:00

🔬 神经网络与反向传播的起源

第一段主要讲述了科学家杰夫·辛顿（Jeff Hinton）与反向传播算法的关联。辛顿在1980年代回到圣地亚哥进行学术休假，并与他人共同创立了PDP（Parallel Distributed Processing）研究小组，专注于神经网络的学习机制。辛顿提出了一种新的方法论，即通过定义问题并调整连接权重来解决问题，而不是单纯从生物学中寻找灵感。这种方法最终被称为梯度下降，其中罗梅尔哈特（Rummelhart）和辛顿共同推广了这一算法，即反向传播算法，它通过从输出层反向传播误差信号来调整隐藏层的连接权重。

05:01

🤖 辛顿的直觉式思考与算法创新

第二段深入讨论了辛顿的思考方式和他对人工智能领域的贡献。辛顿以其直觉式思考和创新算法而闻名，其中包括1981年关于变换器的构想和一篇早期的语义认知论文。辛顿在1977年未发表的论文中提出了一种神经网络进行递归计算的方法，这种方法通过快速改变连接权重来保存和恢复计算状态。此外，辛顿的思考方式对人类数学认知的理解产生了深远影响，他倾向于用图形和直观的比喻来解释深度学习的原理，而不是依赖复杂的方程式。

10:02

🧠 辛顿对计算智能的贡献

第三段探讨了辛顿对计算智能领域的贡献，特别是他在早期至中期80年代对玻尔兹曼机的研究。辛顿试图将布尔逻辑传统与连续概率、分级约束满足领域相结合，以探索推理的本质。辛顿的工作不仅连接了逻辑和理论物理，还启发了对认知和智能更深层次的理解。此外，辛顿的思考方式和对计算智能的贡献被认为是跨学科研究的典范，他的工作推动了对智能本质的探索，超越了人类自身的生物学限制。

15:04

🌟 计算智能的未来与人类智能的扩展

第四段讨论了计算智能的未来，以及如何通过构建不受生物学限制的计算智能来扩展人类智能。提到了谷歌大脑、OpenAI和DeepMind等机构在大规模计算方面的成就，如在围棋游戏和蛋白质折叠问题上的突破。尽管目前人工智能在神经元和突触的数量上还未超过人脑，但它们的发展速度非常快，正在迅速追赶。这一段强调了计算智能的潜力，以及它可能带来的对人类智能的增强和扩展。

Keywords

💡反向传播

反向传播（Backpropagation）是一种在神经网络中用于训练模型的算法。它通过计算损失函数关于网络参数的梯度，并利用这些梯度来更新网络的权重。在视频中，反向传播算法被称为“generalized delta rule”，是梯度下降法的一种扩展，用于多层神经网络的训练。

💡梯度下降

梯度下降是一种优化算法，用于最小化目标函数。在深度学习中，它用于调整神经网络的权重，以减少预测与实际值之间的误差。视频中提到，梯度下降在连续神经网络中已经被理解，并且与反向传播算法紧密相关。

💡杰弗里·辛顿（Jeff Hinton）

杰弗里·辛顿是人工智能领域的先驱之一，以其在神经网络和深度学习方面的贡献而闻名。视频中提到，辛顿在1981年发表了关于变换器（transformers）和语义认知的开创性论文，对深度学习的发展产生了深远影响。

💡玻尔兹曼机

玻尔兹曼机是一种随机神经网络模型，能够学习输入数据的概率分布。它结合了逻辑和统计力学的概念，用于模拟物理系统的热动力学行为。视频中提到，辛顿在早期到中期80年代对玻尔兹曼机的研究，是连接布尔逻辑传统和概率、连续约束满足问题的一种方式。

💡深度学习

深度学习是机器学习的一个分支，它使用多层神经网络来模拟人类学习过程，解决复杂的模式识别问题。视频中讨论了深度学习在图像分类、语言处理等方面的应用，并提到了梯度下降和反向传播在深度学习中的重要性。

💡PDP研究小组

PDP研究小组是由杰弗里·辛顿、詹姆斯·马丁和大卫·鲁梅尔哈特等人组成的研究团队，他们共同推动了并行分布处理（Parallel Distributed Processing）理论的发展。视频中提到，这个小组的成立标志着深度学习和神经网络研究的一个重要里程碑。

💡激活模型

激活模型是神经网络中用于处理信息的模型，它决定了神经网络如何处理输入信号并产生输出。视频中提到的交互激活模型（interactive activation model）是PDP研究小组早期工作的一部分，展示了神经网络如何处理复杂的认知任务。

💡

💡损失函数

损失函数（Loss Function）是衡量神经网络预测值与真实值之间差异的函数，用于指导网络权重的调整。在视频中，损失函数与梯度下降和反向传播算法紧密相关，通过最小化损失函数来优化神经网络的性能。

💡目标函数

目标函数（Objective Function）是优化问题中希望最大化或最小化的函数。在深度学习中，目标函数通常是损失函数，用于指导网络学习。视频中提到，辛顿强调了定义目标函数并调整连接权重以解决特定问题的重要性。

💡计算智能

计算智能（Computational Intelligence）是一个广泛的研究领域，涵盖了机器学习、人工智能、神经网络等，旨在模拟或扩展人类智能。视频中提到，这个领域不仅关注人类智能，还探索超越人类认知限制的可能性。

💡递归计算

递归计算是一种在程序中重复调用同一过程或函数的技术，用于解决需要重复处理数据的问题。视频中提到，辛顿在1977年的一篇未发表的论文中描述了神经网络如何进行递归计算，这是他对深度学习和认知计算贡献的一部分。

Highlights

杰夫·辛顿（Jeff Hinton）作为博士后参与了会议的组织，后来成为助理教授，并有机会回到圣地亚哥进行学术休假。

辛顿、赫特（Heart）和作者三人一起创立了PDP研究小组，专注于神经网络的研究。

辛顿提出应该从问题解决的角度出发，而不是仅仅依赖生物学启发来调整连接权重。

辛顿、赫特和作者开始思考如何通过定义目标函数和梯度下降来最小化误差或损失。

梯度下降在连续神经网络中的概念已经被工程学界所熟悉，其中delta规则由斯坦福大学的伍德罗·伯尼（Woodrow Bernie）和霍夫（Hoff）发明。

罗梅尔哈特（Rommelhart）推广了delta规则，使其能够改变隐藏层单元之间的连接。

辛顿当时认为推广的delta规则可能效果不佳，并有自己的替代算法。

最终，推广的delta算法被命名为反向传播（Back Propagation），因为它通过从输出层向隐藏层反向传播误差信号来调整权重。

辛顿引入了定义目标函数和梯度下降的概念，从而优化和计算连接权重。

辛顿在1981年发表了两篇开创性的论文，一篇关于变换器（transformers）的概念，另一篇关于语义认知的早期研究。

辛顿在1977年未发表的一篇论文中描述了神经网络如何进行递归计算。

辛顿的思考方式对作者理解人类数学认知产生了深远影响，他倾向于用直观的图示而非方程式来解释深度学习的原理。

辛顿的思考方式与费曼（Feynman）类似，结合了视觉和直观的元素，对工程领域的人如吉姆·凯勒（Jim Keller）产生了影响。

辛顿希望贡献于推理本身的理解，而不仅仅是人类推理，他将布尔逻辑传统与连续概率、分级约束满足领域相结合。

辛顿在早期到中期80年代关于玻尔兹曼机的工作是他连接布尔传统的方式，也是将逻辑与理论物理相结合的产物。

作者认为人类是智能的一种表现形式，而智能的认知机制比人类本身更为宏大。

作者提出了“计算智能”（computational intelligence）一词，用以描述像辛顿和DeepMind等研究人员所工作的领域。

深度学习领域的兴奋之处在于，我们可能能够构建超越人类神经系统限制的计算智能，从而扩展人类智能的机制。

谷歌大脑（Google Brain）、OpenAI和DeepMind等正在进行的大规模计算展示了AI在围棋游戏和蛋白质折叠问题解决方面取得的进展。

Transcripts

but just to say something more about

the scientist and and the back

propagation idea that you mentioned um

so

in in nineteen

hinton had been there as a postdoc and

organized that conference he'd actually

gone away and gotten an assistant

professorship and then

um there was this opportunity to bring

him back so jeff hinton was back

on a sabbatical san diego in san diego

and uh remember heart and i had decided

we wanted to do this

you know we thought it was really

exciting and

um our the papers on the interactive

activation model that i was telling you

about had just been published and we

both sort of saw a huge potential for

this work and

and and jeff was there and so the three

of us

uh

started a research group which we called

the pdp research group

and uh several other people

came um francis crick who was at the

sock institute heard about it from jeff

um and uh because jeff was known among

brits to be brilliant and francis was

well connected with his british connect

friends so

francis crick came and a heck of a group

of people wow okay and uh

uh several as paul spalensky um was one

of the other postdocs he was still there

as a postdoc

and um

a few other people but anyway

jeff

talk to us about

learning

and

how we should think about

how

you know learning occurs in a neural

network and he said

the problem

with the way you guys have been

approaching this is that you've been

looking for inspiration from biology

to tell you how

what the rules should be for how the

synapses should change the strengths of

their connections how the connections

should form

he said that's the wrong way to go about

it what you should do is you should

think in terms of

how you can

adjust connection weights

to solve

a problem

so you define your problem

and then you figure out

how the adjustment of the connection

weights will solve the problem

and

removal heart heard that

and

said to himself okay so i'm going to

start thinking about it that way i'm

going to

essentially

imagine that i have some objective

function

some goal of the computation i want my

machine to correctly classify all of

these images

and i can score that i can measure how

well they're doing on each image and i

get some measure of law error or loss

it's typically called in deep learning

and um

i'm going to figure out how to adjust

the connection weights so as to minimize

my loss or reduce the error

uh and

that's called

you know gradient descent

and uh

engineers were already uh familiar with

the concept of gradient descent

uh and in fact um

there was an algorithm

called the delta rule

that had been invented

by

a professor in the engineering

electoral engineering department at

stanford

woodrow bernie woodrow and a

collaborator named hoff i don't never

met him anyway so

so

gradient descent in continuous

neural networks with multiple

neuron-like processing units was already

understood

for a single layer of connection weights

we have some inputs over a set of

neurons we want the output to produce a

certain pattern

we can define the difference between our

target and what the narrow network is

producing and we can figure out how to

change the connection weights to reduce

that error

so what rommelhard did was to generalize

that

so as to be able to change the

connections from earlier layers of units

to the ones

at a hidden layer between the input and

the output

and

so he first called the algorithm the

generalized delta rule because it's just

an extension of the gradient descent

idea

and interestingly enough

hinton was thinking that this wasn't

going to work very well

so hinton had his own alternative

algorithm at the time

based on uh the concept of the balsa

machine that he was pursuing so the

paper on the balsa machine came out in

learning in bolster machines came out in

1985

but it turned out that

backprop worked better than the bolster

machine learning algorithm

so this generalized delta algorithm

ended up being called back propagation

as you say back prop yeah

and

the you know probably that name is

opaque to maybe what what does that mean

what it what it meant was that in order

to figure out what the changes you

needed to make to the

connections from the input to the hidden

layer

you had to

back propagate the error signals

from the output layer through the

connections

from the hidden layer to the

output to get the signals

that would be the error signals for the

hidden layer

and that's how rimmel hard formulated it

was like well we know what the air

signals are at the output layer let's

see if we can get a signal at the hidden

layer that tells each hidden unit what

its air signal is essentially so it's

back propagating through the connections

from the hidden to the output to get the

signals to tell the hidden units how to

change their weights from the input and

that's why it's called back problems

yeah but uh so it

came from hinton having

introduced the concept of you know

define your objective function figure

out how to

take the derivative so that you can um

adjust the connections so that they make

progress towards your goal so stop

thinking about biology for a second and

let's start to think about optimization

and computation yeah a little bit more

so

what about

jeff hinton

what um

you've gotten a chance to work with him

in that little

the set of people involved there is

quite incredible the small set of people

under the the pdp flag

it's just given the amount of impact

those ideas have had over the years it's

kind of incredible to think about but

you know

just like you said uh like yourself

jeffrey hinton is seen as one of the

not just like a seminal figure in ai but

just a brilliant person just a like the

horsepower of the mind is pretty high up

there for him because he's

just a great thinker so what kind of

ideas have you

um

learned from him have you influenced

each other on have you debated over what

stands out to you

in in the in the full space of ideas

here

at the intersection of computation and

cognition

well

so um

jeff has said many things to me that had

a profound impact on my thinking

um and he's written several articles

which

were way ahead of their time

he

he had two papers in 1981 just to give

one example

one of which was essentially the idea of

transformers

and another of which

was a

early paper on semantic cognition which

inspired

him and rommel hart and me

[Music]

throughout

the 80s and

uh um

you know still uh i think sort of

grounds my own thinking about

um the semantic aspects of of cognition

um

he also

uh in a in a small paper that was never

published that he wrote in 1977 you know

before he actually arrived at ucsd or

maybe a couple of years even before that

i don't know

uh when he was a phd student he he um

described how a neural network could

uh do recursive computation

and um

uh

it was a very clever idea that he's

continued to explore over time which was

sort of the idea that

um

when you when you call a subroutine you

need to save the state that you had

when you called it so you can get back

to where you were when you're finished

with the subroutine and and the idea was

that you would save the state

of the calling routine by making fast

changes to connection weights

and then

when you

finished with the subroutine call those

fast changes in the connection weights

would allow you to go back to where you

had been before

and reinstate the previous context so

that you could continue on with the

the

the top level of the computation

anyway that was part of the idea and um

i always thought okay that's really you

know he just

he had extremely creative ideas that

were

quite a lot ahead of his time and

many of them in the 1970s and early

early 1980s

so

another thing about jeff hinton's way of

thinking which um has profoundly

influenced my

um

uh effort to understand human

mathematical cognition

is

that he doesn't write too many equations

and people tell stories like oh in

in the hints and lab meetings you don't

get up at the board and write equations

like you do in everybody else's machine

learning lab

what you do is you draw a picture

and and you know he he explains

aspects of the way deep learning works

by

putting his hands together and showing

you the shape of a ravine

and um

using that as a geometrical metaphor for

the

what's happening as this gradient

descent process you're coming down the

wall of a ravine if you take too big a

jump you're gonna jump to the other side

and um so that's why we have to turn

down the learning rate uh for example um

and

it it

speaks to me of the

fundamentally intuitive character of

deep insight

together with

a commitment to really understanding

um

in a way that's

absolutely ultimately explicit and clear

uh

but also intuitive

yeah there's certain people like that

here's an example

some kind of weird mix of uh visual

and intuitive and all those kinds of

things feynman is another example

different style thinking but very unique

and when you when you're around those

people for me in the engineering realm

uh there's a guy named jim keller

who's a chip designer engineer

every time i talk to him

it doesn't matter what we're talking

about just

having experienced that unique way of

thinking transforms you and makes your

work much better

and that's that's the magic you look at

daniel kahneman you look at the great

collaborations throughout the history of

science

that's the magic of that it's not always

the exact ideas that you talk about

but it's the process of generating those

ideas being around that spending time

with that human being you can come up

with some brilliant work especially when

it's cross-disciplinary as it was a

little bit in your case yeah jeff

yeah um

jeff is uh a descendant of

the logician bool

he comes from a long line of

english

and

together with the

um

deeply intuitive thinking ability that

he has he also

um

has

you know it's been clear he's he's

described this to me um and i think he's

mentioned it from time to time in other

interviews with that he's had with

people um you know he's

he's wanted to be able to sort of think

of himself as contributing to the

to the

understanding of

reasoning itself not just human

reasoning like bull like is about logic

right it's about

what can we conclude from what else and

how do we formalize that and

um as a computer scientist uh

logician

philosopher you know um the goal is to

understand how we derive truths from

other

from givens and things like this and and

the work that jeff was doing in the

um early to mid 80s um

on something called the boltzmann

machine was uh his way of

connecting with that boolean tradition

and bringing it

into the more continuous probabilistic

graded constraint satisfaction realm

um and it was

it was um

beautiful uh

a set of ideas linked with theoretical

physics um and

um

as well as with logic and um

it it's always been i mean i've always

been inspired by the bolson machine too

it's it's like well if the neurons are

probabilistic rather than

you know deterministic in their

computations then

you know that that maybe this uh somehow

is part of the

um

serendipity or you know

adventitiousness of the moment of

insight right

the it it might not have occurred at

that particular instant it might be sort

of partially the result of a stochastic

process

and uh

and and that too is part of the magic of

the emergence of uh

some of these things well you're right

with the bullying lineage and the the

dream of computer science

is uh somehow

i mean i certainly think of humans this

way that humans are one particular

manifestation of intelligence

that there's something bigger going on

and you're trying to you're hoping to

figure that out

the mechanisms of intelligence the

mechanisms of cognition are much bigger

than just humans yeah so i think of um

i've i started using the phrase

computational intelligence at some point

as to characterize the

the field that i thought you know people

like jeff hinton

um and many of the

of the people i know at deepmind um

are are working in and where i i feel

like i'm

um

you know i'm a i'm a kind of a

human-oriented

computational intelligence researcher in

that i'm actually kind of interested in

the human solution

but at the same time i

uh

i feel like

that's that's where um a huge amount of

the

the excitement of deep learning actually

lies is in the idea that

you know we may be able to even go

beyond what we can achieve with our own

nervous systems when we

build

um computational intelligences that are

um

you know not limited in the ways that we

are by our own biology perhaps allowing

us to scale the very mechanisms of human

intelligence just increase its power

through scale

yes

and and i think that that you know

obviously that's the

that's being played out massively at

google brain at open ai and to some

extended deep mind as well um

i guess i shouldn't say to some extent

uh the the the massive scale of the um

computations that uh

are used to

succeed at games like go or to solve the

protein folding problems that they've

been solving and so on still not as many

uh synapses and neurons as the human

brain so we still got

we're still still beating them on that

we humans are beating the ais but

they're catching up pretty quickly

you