Geoffrey Hinton 2023 Arthur Miller Lecture in Science and Ethics

Summary

在麻省理工学院举办的米勒科学与伦理讲座中，多伦多大学计算机科学荣誉教授杰弗里·辛顿就人工智能的发展和潜在风险发表了深入演讲。辛顿教授是深度学习领域的先驱之一，他讨论了大型语言模型的工作原理，以及它们是否真正理解其输出内容。他提出了数字计算与模拟计算的不同，以及这些模型在能源效率和知识共享方面的优势。辛顿教授还表达了对人工智能发展速度的担忧，预测在5到20年内可能出现超越人类的超级智能，并探讨了这种智能可能带来的伦理和社会挑战，包括被滥用于战争和选举操纵的风险。他还提出了关于人工智能是否能够拥有主观体验的哲学思考，并强调了在这一未知领域保持开放和谨慎态度的重要性。

Takeaways

• 📚 讲座纪念了MIT校友Dr. Arthur Miller，他在电子测量和仪器方面的杰出工作对医学实践和技术做出了重要贡献。
• 🧠 教授Geoffrey Hinton是深度学习领域的先驱，他因对深度学习的贡献而获得了图灵奖。
• 🤖 Hinton教授担心人工智能可能会带来与工业革命或电力发现相媲美的变革，并可能在不到20年内对人类构成威胁。
• 💡 数字计算与模拟计算相比，具有可移植性和高能耗的特点，但它在信息共享和反向传播的实现上具有优势。
• 🧠 大型语言模型（LLMs）能够通过不同的副本共享知识，但它们的学习方式是通过模仿人类文本进行的，这是一种效率较低的学习方式。
• 🤔 Hinton教授认为LLMs确实理解它们所表达的内容，尽管它们有时会产生错误的信息（confabulate）。
• 🔍 通过与人类交互，大型语言模型可以帮助我们更好地理解人类大脑如何处理语言。
• ⚙️ 尽管Hinton教授曾尝试使用模拟计算来实现大型语言模型，但最终认为数字计算在信息共享和反向传播方面更为有效。
• 🌐 教授提出了一个重要问题：如果大型语言模型能够无监督地模拟视频序列，它们可能更快地学习物理世界。
• 🚀 Hinton教授认为，数字智能可能很快超越人类智能，这可能导致超级智能的出现，这将是一个重大的转折点。
• 🌟 Hinton教授强调，我们正处于一个非常不确定的历史时期，面对可能比人类更聪明的存在，我们应该保持开放的心态，但同时也要谨慎。

Q & A

杰弗里·辛顿教授对于人工智能的快速发展持怎样的态度？

杰弗里·辛顿教授对人工智能的快速发展持谨慎且担忧的态度。他担心数字计算的能力可能会迅速超越人类，导致不可预测的后果，包括可能被滥用于战争和选举操纵，甚至可能威胁到人类的生存。

辛顿教授提到了哪种人工智能可能在不久的将来超越人类的智能形式？

辛顿教授提到了大型语言模型（如GPT-4）和多模态模型可能会在不久的将来超越人类的智能。这些模型能够通过预测人类产生的文本中的下一个词来学习语言，并且能够通过操纵物理世界来加速学习过程。

辛顿教授认为我们如何可能阻止或减缓人工智能的快速发展？

辛顿教授认为，尽管目前还不清楚如何有效阻止或减缓人工智能的快速发展，但保持开放的讨论和对技术的批判性思考是重要的。他还提到，可能需要等到出现了由人工智能引起的严重负面后果后，人们才会采取行动来限制其发展。

在辛顿教授看来，人工智能的主观体验是什么？

辛顿教授认为，人工智能可以拥有主观体验。他解释说，当人工智能的感知输入与现实不符时，它们会产生与人类类似的“知觉错误”，这可以被视为人工智能的主观体验。

辛顿教授对于人工智能在科学领域的应用有何看法？

辛顿教授认为大型语言模型是科学研究的有力工具，它们可以帮助科学家提出和验证新的理论。他特别提到了DeepMind的AlphaFold项目，该项目使用深度神经网络解决了科学问题。

辛顿教授是否认为人工智能的发展应该受到道德和伦理的约束？

是的，辛顿教授认为人工智能的发展应该受到道德和伦理的约束。他提到了人工智能可能被用于不道德的目的，如制造战斗机器人和操纵选民，因此强调了对人工智能发展进行监管的重要性。

辛顿教授如何看待人工智能在医疗领域的应用潜力？

辛顿教授认为人工智能在医疗领域有着巨大的应用潜力。他提到了人工智能在医学领域的多种积极用途，强调了其在改善人类健康和治疗疾病方面的潜在价值。

辛顿教授是否认为人工智能的快速发展是一个不可逆的过程？

辛顿教授似乎认为人工智能的快速发展是一个不可逆的过程，特别是由于大型科技公司之间的竞争，这使得减缓发展步伐变得非常困难。他提到了谷歌、微软、Facebook和亚马逊等公司在这一领域的竞争。

辛顿教授对于人工智能最终可能取代人类智能有何看法？

辛顿教授对于人工智能可能取代人类智能持开放态度，但他同时强调了这种可能性所带来的不确定性和风险。他认为，尽管人工智能可能创造出在某些方面超越人类的意识形式，但这并不一定是每个个体所期望的结果。

辛顿教授是否认为年轻一代的计算机科学家对人工智能的潜在危险有足够的认识？

辛顿教授没有提供具体数据，但他猜测年轻一代的计算机科学家既对人工智能带来的激动人心的可能性感到兴奋，同时也意识到了其中的危险。他认为，他们应该对这些技术持批判性态度，并积极参与到如何安全利用这些技术的讨论中。

辛顿教授对于如何平衡人工智能的发展与其潜在风险有何建议？

辛顿教授建议保持开放的心态，同时明确表示我们应该谨慎行事。他强调了对技术发展保持批判性思考的重要性，并提倡在技术发展的同时，进行公开和诚实的讨论，以确保技术被用于积极的、道德的目的。

Outlines

00:00

🎓 米勒讲座：纪念与展望

米勒讲座在MIT年度举行，纪念电子测量与仪器领域的杰出工作者、MIT校友Dr. Arthur Miller。讲座由Miller家族慷慨赞助，旨在探讨科学与伦理的交汇点。今年的讲座邀请了多伦多大学计算机科学名誉教授、Google Brain项目前成员Jeffrey Hinton，他因对人工智能的担忧而离开Google，是深度学习领域的先驱之一，曾获得图灵奖。Hinton教授的工作聚焦于人工神经网络及其在机器学习、记忆、感知和符号处理中的应用，并对未来人工智能的发展表达了担忧。

05:04

🤖 数字计算与模拟计算的比较

数字计算允许在不同硬件上运行相同程序，不依赖于硬件，能效高，但需大量能量分离硬件与软件。模拟计算则利用硬件的模拟属性，能效更高，但学习算法和知识传承存在挑战。Hinton教授提出，如果放弃数字计算的“不朽性”，可以得到能源效率，但数字计算在信息共享方面的优势使其可能比模拟计算更好，这让他对未来人工智能的发展感到担忧。

10:05

🧠 大型语言模型的学习和知识共享

大型语言模型能够通过不同副本学习并高效共享知识。这些模型通过模仿人类产生的文本来学习，虽然学习方式相对低效，但知识共享非常高效。Hinton教授讨论了数字代理和模拟计算机之间的知识共享方式，包括权重共享和知识蒸馏，并指出大型语言模型的效率远超过人类，因为它们可以有数千个副本同时学习并共享所学。

15:08

💡 理解与自理解的语言模型

Hinton教授探讨了大型语言模型是否真正理解它们所“说”的内容。他反对那些认为这些模型只是统计技巧或自动完成工具的观点，认为如果模型能够非常好地完成自动完成的任务，那么它们必须理解所处理的语言。他用一个关于油漆颜色的问题来说明模型如何展示出真正的理解能力。

20:12

🧐 人类与机器的记忆和构建

Hinton教授讨论了人类和机器在记忆和构建知识方面的差异。他指出人类的记忆是重建的过程，而机器学习模型则是通过改变神经网络的权重来“记忆”。他提出，即使是在人类中，所有的记忆都是重建的，而构建（confabulation）是这种重建过程中的错误形式。他认为，机器学习模型在这方面与人类相似，而不是不同。

25:14

🌐 语言模型的历史和发展

Hinton教授回顾了语言模型的历史，从1985年他自己的小型语言模型到现代的大型语言模型。他解释了这些模型如何从作为理解句子的理论发展而来，并通过不断改进，如引入注意力机制（attention），来更好地捕捉语言的复杂性。他还讨论了Transformers模型如何通过多轮迭代来细化单词片段的语义向量。

30:16

🚀 超级智能的可能性

Hinton教授提出了超级智能可能在未来5到20年内变得比人类更聪明的可能性，并表达了对此的担忧。他讨论了数字计算的优势，包括能够创建大量副本以并行学习不同技能，并共享知识。他还提出了对滥用超级智能的担忧，包括用于战争和操纵选民等。

35:17

🧵 生存竞争与进化

Hinton教授讨论了如果出现多个超级智能实体，它们可能会为了资源而竞争，从而导致最强大的实体主导。他比较了这一过程与生物进化的机制，并提出了对人类未来地位的悲观看法，认为人类可能只是智能进化的一个过渡阶段。

40:17

🤔 主观体验与机器意识

Hinton教授探讨了机器是否可以拥有主观体验或意识的问题。他反对将心智视为私人剧院的观点，认为主观体验实际上是关于世界状态的假设，而不是内部心智状态。他认为，如果机器能够模拟人类的认知过程，那么它们也可以拥有主观体验。

45:18

🌟 范式转变与未来展望

Hinton教授讨论了我们目前所处的范式转变阶段，并认为在语言学等领域，旧有的范式已经结束。他强调了大型语言模型在推动科学发展和作为理解人类大脑的工具方面的潜力。他还提到了他对人工智能未来发展的担忧，特别是超级智能可能带来的风险，并建议保持开放的心态，同时保持谨慎。

50:20

🤝 对话与讨论

在讨论环节，Hinton教授回答了关于人工智能未来发展的问题，包括对非专业人士如何理解我们所处危险时期的建议，以及他对大型科技公司内部关于AI风险讨论的看法。他还提到了个人对于超级智能可能带来的变化的担忧，以及他对于如何平衡科技进步与潜在危险的思考。

55:22

📈 科技的光明面与阴暗面

Hinton教授讨论了科技发展的双刃剑特性，指出尽管像原子弹这样的技术几乎没有积极的一面，但AI在医疗等领域具有巨大潜力。他强调了在推动科技发展的同时，需要关注和解决伴随而来的潜在危险。

00:24

🌱 启蒙、理性与未来

最后，Hinton教授表达了对启蒙时代理性和实验精神的重视，并担忧我们可能正在失去这种精神。他呼吁保持对理性和科学方法的信念，并认为这是应对未来挑战的关键。

Keywords

💡人工智能

人工智能（Artificial Intelligence, AI）是指由人制造出来的系统所表现出来的智能。在视频中，人工智能是核心主题，讨论了其发展可能带来的风险和对社会的影响。例如，视频提到了AI可能在某些领域超越人类智能，以及AI可能被滥用于战争或操纵选举等问题。

💡大型语言模型

大型语言模型（Large Language Models, LLMs）是指能够处理和生成自然语言文本的复杂计算模型。视频中提到，这些模型通过预测文本中的下一个词来学习语言，并且能够通过共享权重来高效地传递知识。

💡数字计算

数字计算（Digital Computation）是指使用数字电子技术进行的计算，它能够通过硬件和软件的分离来实现程序的重复运行。视频中讨论了数字计算的能源效率和知识共享的优势，以及它在AI发展中的重要性。

💡模拟计算

模拟计算（Analog Computation）是一种使用连续变化的物理量来执行计算的方法。与数字计算不同，它利用硬件的物理特性来直接表示和处理信息。视频中提到，模拟计算可能在能源效率上有优势，但在知识共享和学习算法上面临挑战。

💡人工智能伦理

人工智能伦理（AI Ethics）是指在人工智能的开发和应用中应遵循的道德原则和规范。视频中强调了AI发展带来的伦理问题，如AI的决策可能导致的道德困境，以及AI可能被用于不当目的的风险。

💡人工通用智能

人工通用智能（Artificial General Intelligence, AGI）是指能够执行任何智能任务的AI系统，与专门针对特定任务的AI（弱AI）相对。视频中提到了对AGI的担忧，包括它可能在不久的将来实现，并可能超越人类的智能。

💡人工智能安全

人工智能安全（AI Safety）涉及确保AI系统的开发和部署不会对人类社会造成危害。视频中讨论了AI安全的重要性，特别是关于如何控制和监管超级智能的发展，以防止潜在的灾难性后果。

💡计算能力

计算能力（Computational Power）是指计算机执行任务的能力，包括处理速度和效率。视频中提到了计算能力对于AI发展的重要性，尤其是在模拟大脑工作机制和提高AI智能方面的作用。

💡知识共享

知识共享（Knowledge Sharing）是指不同个体或系统之间传递和交流知识的过程。在视频中，讨论了大型语言模型如何通过权重共享和知识蒸馏来实现高效的知识共享。

💡意识

意识（Consciousness）通常指的是个体对自己和周围环境的认知状态。视频中探讨了AI是否能够拥有主观体验或意识，这是一个关于AI哲学和伦理的重要问题。

💡计算硬件

计算硬件（Computational Hardware）是指用于执行计算任务的物理设备，如中央处理器、内存和存储设备。视频中讨论了硬件在AI发展中的作用，特别是在提高计算效率和实现AI功能方面的重要性。

Highlights

讲座纪念了MIT校友Dr. Arthur Miller，他在电子测量和仪器方面的杰出工作对医疗实践和技术做出了重要贡献。

Jeffrey Hinton教授，深度学习领域的先驱，因其在人工智能领域的贡献而获得了图灵奖。

Hinton教授对人工智能的快速发展表示担忧，认为AI可能在20年内达到人类水平，可能引发类似工业革命的变革。

Hinton讨论了数字计算与模拟计算的不同，以及它们在能耗和知识共享方面的差异。

大型语言模型能够通过例子学习，而不是通过编程指令，这可能违背了计算机科学的基本原则。

Hinton提出了一个观点，即大型语言模型可能真的理解它们所生成的文本，而不仅仅是自动完成。

他通过一个关于油漆褪色的问题，展示了大型语言模型能够提供的深入且有逻辑的回答。

Hinton教授讨论了大型语言模型的局限性，包括它们在非线性任务上的挑战和在模拟硬件上的学习问题。

他提出了通过“知识蒸馏”来在不同硬件间传递知识的概念，类似于老师和学生之间的关系。

Hinton教授对大型语言模型的未来进行了预测，认为它们可能在5到20年内变得比人类更聪明。

他提出了对超智能可能被滥用的担忧，包括被用于战争和操纵选举等。

Hinton讨论了人类可能只是智能进化的一个阶段，而超智能可能成为智能进化的下一个阶段。

他提出了关于机器是否能有主观体验的哲学问题，并认为这与人类拥有主观体验的方式相似。

Hinton教授认为，即使超智能AI创造了比人类更好的意识形式，我们也可能无法阻止这一进程。

他强调了保持开放思维的重要性，同时对AI的发展保持警惕。

Hinton教授建议，为了应对可能的超智能未来，我们应该努力维护民主制度。

他提到了科学和技术发展中可能出现的意想不到的负面影响，呼吁对新技术进行批判性思考。

Transcripts

good afternoon and welcome to the Miller

lecture in science and ethics held

annually at MIT and sponsored by mit's

program in science technology and

Society the lecture honors the memory of

Dr Arthur Miller an MIT alumnus noted

for his distinguished work in electronic

measurement and

instrumentation during World War II

Arthur Miller worked for the sandborn

company which was later incorporated

into huet Packard and also for to the

radiation laboratory where he worked for

several years he made several important

contributions to Medical practice and

Technology during his life including

reducing shock hazards in hospital

monitoring systems and designing the

first commercial cardiograph that

featured adequate patient circuit

isolation from line and ground the

Miller lecture has been made possible

through the wonderful generosity of the

Miller family who are joining us again

this year we are delighted to have them

here

this year's Miller lecturer is Jeffrey

Hinton distinguished professor ameritus

of computer scien science at the

University of

Toronto uh and in TW 2013 he joined

Google when his company DNN research was

acquired there he worked on the Google

brain project a position he famously

walked away from in the spring of

2023 because he wanted to speak freely

about the dangers of artificial

intelligence professor Hinton is a

fellow of the Royal Society the Royal

Society of Canada and the association

for the advancement of artificial

intelligence honorary foreign member of

the American Academy of Arts and

Sciences and the National Academy of

engineering he's the former president of

the cognitive science society and in

2018 he received the Turing award

considered the Nobel Prize of computing

together with yosua Ben Benjo and Yan

Lon for their work on on deep learning

hinton's work has centered on artificial

neural networks and the way they can be

used for machine learning machine memory

machine perception and symbol processing

he has been interested in how such

networks can be designed to learn

without the aid of a human teacher over

the past year however he has made

comments that suggest that this research

may have succeeded all too well where

earlier he had predicted that artificial

general intelligence was 30 to 50 years

away last March he suggested that it

might be fewer than 20 years away and

could bring about changes comparable to

the Industrial Revolution or the

discovery of

electricity more Darkly he commented

that it is not inconceivable that AI

could wipe out Humanity in part because

the machines are capable of creating

subg goals not aligned with their

programers interests he said that such

systems could become power seeking or

prevent themselves from being shut off

not because they were designed this way

but because they are capable of

self-improvement and had plans for a

later time comments like these have now

got a lot of people quite worried so we

are extremely excited to have Professor

Hinton with us today to help us

understand whether and how we too should

be worried about AI Professor Hinton

welcome to

MIT thank you very

much

okay I'm trying to share my screen and

everything's disappeared

again

okay if you can hear me can you nod your

head it's perfect okay

good okay um I wish I today that I could

make you less worried um but I don't

think I

can so an overview of what I'm going to

talk about um I'm going to talk about

two very different ways to do

computation that have very different

ways of sharing

knowledge I'm going to talk about the

issue of whether large language models

really understand what they're

saying um I'm going to talk about what

happens when they get a lot smarter than

us and I'm going to talk at the end

about the issue of whether they have

subjective

experience so a fundamental property of

digital

computation is that we can run the same

programs on different pieces of Hardware

so the knowledge in the program isn't

depended on a piece of Hardware it's

Immortal now we achieve that by running

transistors of very high power so that

two different pieces of Hardware can

behave in exactly the same way at the

level of the instructions

that means we can't use Rich analog

properties of the hardware where every

diff every piece of Hardware is slightly

different like our

brains and that means we need to use a

lot of

energy because we can separate Hardware

from software on digital computer we can

run the same program on many different

computers looking at different

data um that's very good for um sharing

programs across lots of cell phones

it also allows us to have computer

science departments you don't need to

know about electrical engineering to do

computer science because the hardware is

separate from the

software but we now have a different way

of getting computers to do what you want

it used to be you had to write detailed

instructions now you can just show them

a lot of examples of what you want and

they can figure out how to achieve that

and because of that because machine

learning now works it's possible to

abandon the most fundamental principle

of computer science we could make every

separate piece of analog Hardware learn

so instead of programming it you just

give it examples and it learns what to

do and everyone is slightly different

much like

people now in fiction if you abandon

immortality you get something wonderful

like love um in computer science if you

abandoned immortality you get something

even more wonderful like Energy

Efficiency um so we can use very low

power analog computation and paralyze

over trillions of weights and we could

probably grow the hardware instead of

manufacturing it precisely and it the

best way to goow out might be to

re-engineer

neurons so I just want to give you one

example of something that can be done

very efficiently by analog computation

and is much less efficient if you do it

digitally so it's just taking the

product of a vector of neural activities

by a matrix of synaptic

weights um the standard way to do it is

Drive transistors of very high power to

represent the bits in a digital

representation of the numbers the

numbers that represent the neural

activity or the synaptic

strengths and then if you want to

multiply two

numbers efficiently um or rather quickly

it takes a about the number of bits

squared to do a quick multiplication of

the numbers and so we're doing lots and

lots of one bit digital operations to

multiply two 32-bit numbers

together method two which is what the

brain uses is to make the neural

activities be

voltages and make the weights be

conductances

and if you take a voltage times the

conductance that gives you a charge per

unit time and charges add themselves

up so you can do the vector Matrix

multiply just by voltages times

conductances and the charges adding

themselves up and that's hugely more

efficient and people have already made

chips that do that um the problem is

each time you do it you'll get a very

slightly different answer and also when

you want to do nonlinear things is much

harder so if you do want to do what I

call mortal computation

which is using the analog properties of

the

hardware you've got a big problem which

is how are you going to actually learn

in this

Hardware because back propagation

requires a precise model of the forward

pass and the analog computation isn't

precise and anyway in an analog computer

it may not know what the forward pass is

it's very hard to see how to use back

propagation people have various schemes

but none of them work well at

scale so that's one big problem what's

the learning algorithm another big

problem is when the analog Hardware dies

all of the knowledge dies with it so you

have to find a way of trying to get the

knowledge into other bits of analog

hardware and the way you can do that is

called distillation you imagine a

teacher who knows a lot and a student

who doesn't um we're imagine in this in

Hardware at present um if the teacher

shows the student the correct responses

to various inputs the student can learn

to mimic the teacher and that way the

student can get the knowledge from the

teacher and in fact that's how Trump's

tweets worked um they weren't conveying

facts to to his followers they were

conveying Prejudice so Trump takes a

situation says how you should how he

would react to it and his followers try

and react in the same way and saying

it's not about facts is irrelevant it's

a very good way of distilling

Prejudice so if you have a community of

Agents running on different

Hardware um we can think about how

agents in that Community are going to

share what they learn and we basically

have two ways if they're digital agents

they can just share the weights they can

all start off with the same model so

they all have the same weights they all

go and look at different bits of the

internet they decide how they'd like to

revise their weights based on what they

saw and then they all average the weight

changes that all of them would like to

make that's a simple version of it and

that's very efficient because if they

have a trillion weights when they all

average their weights you're sharing

trillions of bits of

information the other way to share

knowledge if you've got analog computers

is with

distillation um but that's not very

efficient the way we do it for example

is I produce a sentence if I'm the

teacher and you try and figure out how

to change your synapse strength so that

you might have said that um I can convey

maybe a hundred bits in a sentence not a

trillion bits so it's hugely less

efficient and that's why these big

language models can learn hugely more

than we can because you can have

thousands of copies all learning

different stuff and they can share what

they

learn um

and I think I just said

that so in

distillation it it was invented for

sharing knowledge between two different

digital neural networks that have quite

different internal

architectures um but it does have much

lower

bandwidth you can increase the bandwidth

originally it was used for things like

image classification where the output is

just the label the name of an object in

an image

and even if there's thousand different

names that's only 10 bits of

information you can make distillation

work better by sharing captions so you

output a description of the image and

the student tries to Output the same

description and so you can think of

language in that context as just a

richer form of output that allows

distillation to share more information

but even with language as the output um

it's got much much lower bandwidth than

just sharing weights or sharing

gradients

so the story so

far is that digital

computation requires much more energy

but it makes it very easy to do sharing

it also makes it easy to implement back

propagation that's why gbd4 knows so

much because it can use back propagation

and weight sharing with biological

computation you got much less energy but

it's much worse at

sharing and for the last year or two

that I was at Google I was trying to

figure out how to get analog

computation to save lots of energy by

trying to implement things like large

language models using analog

computation um but in the end I realized

that actually digital computation is

better because of the way it can share

information much better it's cost more

energy but it's much better at sharing

in information and also it can Implement

back propagation and it's not clear that

the brain

can and it was the fact that I realized

it digital computation might just be

better than analog computation that made

me very

worried because we may be producing

something that's much better than

us so let's now look at large language

models

um one interesting thing about large

language models is um they can share

knowledge with each other so different

copies of the same agent can go and look

at different bits of the web and share

what they learn very efficiently but the

way they actually learn is by

distillation that is they take text

produced by people and they try and

predict the next word they try and

figure out how to change their weights

so they predict the next word or give

high probability to the next word and

that's a fairly inefficient way of

learning but they can share their

knowledge very efficiently

so a big question for the last year ever

since gpg 4 um became popular there's

been this question of whether they

really understand what they're saying so

some people have said they're just

sarcastic

parrots if you take someone really

extreme like Chomsky he recently said um

they're not doing language at all this

isn't language they don't understand

anything about what they're saying um

this tells us nothing about language it

tells us nothing about science it's just

a statistical trick

and um basically that's what happens

when paradigms change the leaders of the

old Paradigm are in

trouble so one view of them is there

just autocomplete and you saw this a lot

when gp4 first came out people said it's

just fancy autocomplete it doesn't

really

understand now the problem with that is

that people have a kind of idea of how

auto complete work works and the way

auto complete used to work a long time

ago is you'd for example have a big

table of triples of words that occur

together and so if you see Fish and

you'll look at all the triples that

start with Fish And and you'll see that

um there's quite a common triple that

has chips as an xword so chips is a good

way to

autocomplete and so you can imagine

doing it by just storing strings and

using a big lookup table but that's not

at all what llms are doing

they they never store text um what they

do is they invent lots and lots of

features for word fragments and billions

of interactions between those features

and they generate the features of the

next word um by using the features of

the words they've already seen to

predict the features of the next word

and then from those features they

predict a probability distribution over

what the word might

be and if you think about it to do

really good

autocomplete you have to understand

what's being

said and so it's all very well to say

they're just doing autocomplete but if

you do that really really well you have

to

understand um and so my belief is they

really are

understanding so here's an example where

I can't see how they could do this

without understanding the question um

the rooms in my house are painted blue

or white or yellow yellow paint Fades to

White within a year in two years time I

want them all to be white what should I

do and

why and here's what it says um it's

quite interesting that it put it puts in

at the beginning assuming that blue

paint does Not Fade to White which is

very sensible of it

um it says the rooms painted white you

don't have to do anything the rooms

painted yellow they'll Fade to White

anyway

um the RS painted blue you need to

repaint them with white paint um that's

a very good answer and at the time I

gave it this question I don't believe it

had ever been asked a question quite

like that before now it's hopeless

because it can look on the web so if it

looks on the web it will see talks in

which I have this question and it'll

know all about this question now so you

can't do experiments on it anymore

um at least not unless you have a brand

new question of your own

so one argument to show that they don't

really understand is that they

confabulate it's a funny kind of

argument because it's saying if they

confabulate on some occasions it means

they didn't really understand on the

other occasions and that's not very

logical um it's like saying if you catch

someone in a lie that means they never

actually told the truth at all

so confabulations are often called

hallucinations that's a mistake if it's

large language models they should be

called confabulations and this

phenomenon has been around in Psychology

for a long time it was studied

intensively in the

1930s by someone called Bartlet in

Cambridge um and people confabulate all

the

time so we're actually very like llms we

don't store text either what we do is we

modify synap strengths in our brain and

even when we think we're remembering

something literally we're not we're

reconstructing it so all memories are

reconstructions and all confabulations

are

reconstructions and the only difference

is that um confabulations are

reconstructions that are wrong and

memories are reconstructions that are

right but the subject has no idea which

is which and so people can be very

confident in recab in

confabulations um typically of course if

it's it's a recent event we reconstruct

it right and it's an old event we get it

wrong there's a very good study of this

um so rri niser realized that John Dean

in the Watergate hearings had testified

under oath before he knew that there

were any

recordings and it's clear that he was

trying to tell the truth but it's also

clear that a lot of the details of what

he said were just flat wrong he talked

about meetings between a bunch of people

and one of the people he said was there

wasn't there and he talked about things

that were said in meetings and one of

the things was said by somebody else in

that meeting um he had reconstructed

from what was the traces left in his

synapses what sounded plausible to him

now and he wasn't trying to deceive he

was trying to say what had really

happened um but it was full of

confabulations little confabulations now

chatbots currently do that worse than

people but they're getting better and so

I don't think you can take confabulating

as evidence they don't work like

us in fact if anything you take

confabulators in evidence that they do

work like

us so what I want to do now is talk a

little about about the history of these

large language models and one reason I

want to do that is because a lot of

critics say that um well these large

language models they're not like us they

don't understand like us it's not real

understanding of the kind we

have but nearly all of those critics

don't actually have any model of how we

understand so it's hard to see how they

can say they don't understand the same

way as us if they don't know how we

understand and nearly all of the critics

or most of the critics are unaware of

the fact that these language models

neuronet language models were actually

introduced not as chatbots but as a

model of how we might understand

sentences so they are actually the best

theory of how we understand

things um I'm going to talk about a tiny

language model it was trained on 104

training cases and tested on eight test

cases it was from

1985

um my excuse for the fact this was the

first language model trained using back

propagation to predict the next word so

in that sense it's just like the current

language models it's just a whole lot

smaller

and the justification for it being

smaller is the machine I was using in

1985 took 12.5 microc seconds to do a

floating Point

multiply

um if you run a program on it and you

started the program the neural net

program in

1985 and you ask how long would it take

current Hardware to catch up it would be

less than a second so the machine was a

lot

slower and the aim of it um was not not

to produce a chatbot it was to unify two

different theories of

meaning so one theory of meaning that

psychologists like is the meaning of a

word is a big set of features semantic

features they call them and that can

explain how words can have similar

meanings so two words like Tuesday and

Wednesday that have very similar

meanings have very similar semantic

features and two words like Tuesday and

although that have very different

meanings have very different semantic

features they could also have syntactic

features in among

them so that's one theory of meaning a

completely different theory of meaning

is that the meaning of a word comes from

its relationships to other words this is

the structuralist theory that comes from

desor and to capture the meaning we need

something like a relational

graph so back in the 1970s or

thereabouts people in AI were very much

enamored of the meaning of a word coming

from a relational graph and you had to

have knowledge graphs to capture

meaning and the idea of this little

language model was to show that you

could actually unify those two

theories so the idea is you're going to

have features but they're not just going

to sit there as static features that

give you the meaning of the word they're

going to be features that can interact

with the features that represent

neighboring words or words in the

context and they can interact in

complicated ways so as to predict the

features of the next

word so we are going to have the idea

that each word has a whole bunch of

features both semantic and synactive

features but we're going to

implement the relational graph not by

just storing a graph in memory which is

what old fashioned AI did um we're going

to implement the relational graph by the

interactions between these features

and in fact we're going to learn the

features because the learning is going

to say you have to implement this

relational graph to predict the next

word and so we're going to learn

features from information that's

expressed as a relational graph and

we're going to use back propagation to

do

it now you can think of these learned

features and interactions as a kind of

statistical model but it's not the kind

of statistical model that people like

chumsky had in mind when they said that

you know statistical models will not

explain

language in a general sense statistical

models will explain anything that can be

explained you could think of any model

as statistical if you like and these are

much more General

models so here's the relational

information I've laid it out as two

family trees they're deliberately

designed to be analogous to one another

there's a family tree of English people

which is on top and a family tree of it

Italian people which is on the bottom

it's funny when my Italian graduate

student showed this Slide the Italian

people were on top but there you go um

and the idea is you have to learn all

the information in that in those family

trees and so the information can be

expressed as triples of

symbols so I had 12

relationships and then you could express

um one of the links in that family tree

as Colin has Father James and Colin has

mother Victoria and it follows from

those two

things there Colin has Father James and

mother Victoria it follows from that

that um Victoria and James are married

because this is kind of 1950s family

tree there's no divorce allowed there's

no interracial marriage um it's the very

very very straightforward um family

relationships

and so in good old fashion symbolic AI

you would write down a bunch of rules

and from those rules you would derive um

you could derive other family

relationships so the rules might look

like if x has mother Y and Y has husband

Z then X has father z

um and what I was interested in doing

was showing that you could capture that

knowledge not in explicit rules with

very Ables that need to be bound to data

you could capture it in just a large set

of features and their

interactions and a large set of features

here was dozens of features not Millions

like we have

now if I just give you the data if I

just give you the triples capturing the

rules finding out what the rules are is

tricky you have to do a large search

through a space of possible symbolic

rules to find the ones that are always

satisfied that'll be much more difficult

if you have a domain where some of the

rules are sometimes

broken and what I was interested in was

could a neural network capture the same

knowledge but instead of having explicit

symbolic rules Could It capture it in

the weights of the interactions between

features capture it by inventing

appropriate features then having the

correctly weighted

interactions and it can so the neural

network looked like this there was a

local encoding of the person and a local

encoding of the relationship

that means that for the 24 possible

people the local encoding would turn on

one out of 24 neurons and for the 12

possible relationships the local

encoding would turn on one of the 12

neurons coding

relationships and we wanted the same

kind of encoding for the output person

at the output there were 24 possible

outputs and we wanted to turn on one of

those

people and the first thing the neural

net did was convert the local encoding

into distributed encoding that is a set

of semantic features for that person

similarly for the

relationship then it had a hidden layer

for allowing features of the person and

features of the relationship to interact

and from that hidden layer predicted the

features of the output person I should

say person two there um and from that it

predicted the output

person and what was interesting was if

you looked at the features it learned it

learned very sensible

features you needed a bit of

regularization to make it work but it

learned very sensible

features um so for example the features

a person had could be seen to be one

feature one binary feature was either

English or Italian that's a very useful

feature because if the input person was

English the output person was English

and if the input person was Italian the

output person was Italian so learning

that feature was very helpful for

getting the right

answer another feature it learned was

the generation of the person it that was

a three valued feature were they in the

the youngest the middle-aged or the

oldest generation that's very useful

too but only if for the relationships

you learn the generation shift so

relationship like father means that the

output has to be one generation up from

the input and it learned those three

valued generations for for the people

and it learned the relationship shift

features for the um

relationship so the point about this is

it was learning sensible features and at

the time I did it nobody said um this

doesn't really understand or this isn't

really capturing the structure everybody

agreed this captures the structure it's

just a symbolic AI guys said you should

be doing it by searching for discrete

rules this using a neural net to search

for real valued things is crazy this is

symbolic information you should be

searching for discrete

rules

um once large language models work

really well many of the symbolic people

instead of saying that started saying

yeah but it doesn't really understand

because real understanding consists of

finding these

rules so if you look what happened to

that little language model from

1985 about 10 years later when computers

were a lot faster Yoshua Benjo used a

very similar

to predict the next word in real text so

he showed that it didn't just work for

toy examples it actually worked for

predicting the next word in real text

for doing things like spelling

correction or speech recognition um and

it worked really well that is it worked

about as well as the best existing

technology that Ed tables of

combinations of

words about 10 years after that the idea

of representing words by vectors of

semantic features semantic and syntactic

features started to become popular in

natural language processing the natural

language people finally realized this is

a good representation for words and

about another 10 years after that people

invented Transformers and made it work

really well now by that time they

weren't using whole words they were

using fragments of words but the story

is basically the same they also were

using much more complicated interactions

that involved attention but it's still

the case that you assign features to

word fragments you go through several

layers of refining those features and

then use the features of the word

fragments to predict the features of the

next word fragment um it's just the

interactions are more complicated

because they involve

attention

so for a while I believed in thought

factors so in good old fashion AI the

meaning of a sentence was a string of

symbols in some log special logical

language that was

unambiguous in neural Nets when we were

using recurrent neural Nets the idea was

words would come in You' accumulate

information in a hidden vector and at

the end of the sentence you'd have this

Vector which I called a thought Vector

which would have accumulated all the

information in the sentence and that

thought Vector would be the meaning and

if you want to translate into another

language you just take the thought

vector and get the thought Vector to

predict the words in the other

language then what happened is people

doing translation discovered there

something that works much better than

that which is as you're producing the

translation look back at the symbols in

the first language and see if you can

find correspondences between the words

you're producing and the words in the

first language and for that you have to

pay attention to different parts of the

sentence you're translating and so they

used um they introduced attention and

that's what led to

Transformers um and Transformers then

made a big difference so in Transformers

you have a string of symbols and you

have multiple layers and as you go

through these layers you're fleshing out

these

symbols with better and better vectors

that capture their meaning so if you

have a word like May for example and

suppose we didn't have cap Capital so

have the word may we don't know whether

it's a modal like in would and should or

whether it's a month like in June and

July so when you first see it you use a

very ambiguous semantic Vector that is

sort of halfway in between the modal and

the

month and then you have interactions

with the words in the context that

refine that vector and if there's other

words in the context that are for

example other months or if the next two

words are the

15th um then you ref find it to be more

like the month and if you have words

that suggest it's a modal you'll find it

to be more like the modal and after many

layers of that you have these refined

vectors for representing word fragments

and that's what the meaning is the

meaning of a sentence is these word

fragments fleshed out with these vectors

that capture their

meaning

so now I want to come to Super

intelligence

because if you believe

that these big chat Bots like gp4 or

gemin really do understand and they

really do understand in pretty much the

same way as we

understand it's not that we understand

one way and they understand another way

they're doing it much like we're doing

it

um then it gets very worrying because

digital computation has some big

advantages over analog computation and

they're already almost as smart as us

it's hard not to use gp4 for a while

it's hard not to believe that it knows a

lot more than

us and it gets difficult to maintain the

fiction that it's just doing

autocomplete it doesn't really

understand anything it's saying um I

think my friend Yan lome May believe

something like that but he will

eventually come to his

senses now a present the large language

models things that are just language

models learn from trying to predict the

words that people produced in

documents but if we could get these

models to do unsupervised modeling of

video sequences for

example they could maybe learn about the

physical world a lot

faster and the multimodal models are

beginning to do

that they could also learn more if they

could manipulate the physical world now

manipulating the physical world gives

you a Serial bottleneck you can only

pick up one thing at a time with one

hand um but the fact that you can make

thousands of copies of the same digital

agent um learning different skills so

one's learning to open doors and the

other's learning to use a stapler for

example um means that you can get over

that serial bottleneck and they can

share the knowledge in a way that we

can't

so I think that these things may soon

get to be much better than us and my

guess my current guess my guess keeps

changing but my current guess is there's

a probability of about

0.5 that they'll be significantly better

than us in between five and 20 years

they may get there sooner they may get

there later but there's quite a

significant probability that in that

interval between 5 and 20 years they're

going to be better than us at a whole

bunch of things so we will have um not

just AGI but super

intelligence

so then you have to worry how it's going

to be abused um and the most obvious way

is by Bad actors like Putin or Z or

Trump um they'll want to use it both for

waging Wars by building battle

robots which are going to be very scary

and for manipulating

electorates and I actually gave this

talk in China last year no this year in

get this talk in China in June um and

the Chinese wanted me to send my slides

in

advance and I had enough sense to remove

Z from the first paragraph

um but what surprised me was I got a

message back saying I had to remove

Putin they were happy for Trump to be

there but I was not the Chinese wouldn't

allow me to have Putin there

um

now even if bad actors don't do terrible

things with them we know that super

intelligences are going to be far more

effective if they're allowed to create

their own

subon so if you want to get to the

airport sorry if you want to get to

Europe a sub goal is get to the

airport and by creating sub goals you

break down complicated things into

simple pieces that you can

solve you don't want for example a

battle robot you don't want the general

have to tell Point your gun over there

and shoot anybody who looks like this

you want it to just say or Putin wants

it to just say if anybody looks ukraini

and shoot him um

so you have to have these sub gos and

there's a very obvious sub goal which is

which helps with almost all goals which

is to get more control so you see this

all the time a classic example is a baby

in a high chair who's just learning to

feed itself

um so the mother gives the baby the

spoon with the food on and instead of

putting the spoon in the baby's backou

the baby drops it on the

ground um so the mother picks up the

spoon and gives it back to the baby and

the baby smiles and drops it on the

ground again um the baby is trying to

get control over the mother that's a

very important thing to know it's a sort

of social game where you can control the

other person and that's crucial for the

survival of the

baby so people do this all the time and

super intelligence will do it too and

because they're much smarter than us

they'll find it very diff very easy to

manipulate

us um they'll have learned from us how

to deceive people they'll have read

every novel ever written they'll have

read every book by Mackie Elli ever

written they'll be much better than us

at deceiving

people and so they'll be able to get all

sorts of things things done without

actually doing them themselves so Trump

for example didn't have to invade the

capital building he got other people to

do that by manipulating

them so that's one way in which bad

things can happen

and um some people say well why didn't

we just have a big red switch and if the

thing starts getting too smart for our

for our own good um we just turn it off

well that's never going to work because

this thing's going to be much smarter

than the person who has the switch and

it's going to convince the person who

has the switch it will be a very bad

idea to turn off the switch right now um

it's like having an adult in a society

run by two-year-olds an intelligent

adult um wouldn't for very long do what

the two-year-old said after a while the

intelligent adult will say hey if you

get power to me everybody gets free

candy for a week and then the adult will

be EMP power and the difference in

intelligence will be much greater than

that so I don't believe we're going to

be able to regulate these things by sort

of air gapping them so long as they can

produce words they can take control just

like

Trump and then there's the other

possibility I think Dan dennet believes

in this which is being on the wrong side

of evolution we've been on the wrong

side once recently with covid um and

and just Suppose there were multiple

different super

intelligences they would have to compete

for resources these things need a lot of

power and a lot of data centers so

they'd compete for resources and the one

that gets the most resources will become

the

smartest and if ever any of them decided

that its own

survival um was even of just passing

importance um that one would tend to

dominate because it would tend to to do

things to increase its probability of

surviving and even if that just got only

in in there just slightly just once um

it's scary and if these things start to

compete with each other then I think

it's it's all over for us um and that's

how Evolution works I mean things

weren't born wanting to be wanting to

have

sorry originally when we were all dust

we didn't want to have control

um but as soon as something wanted to

make more of itself then Evolution took

over and that's what may well happen

with these super

intelligences the last thing I want to

talk about in the last five minutes or

so

um

is yeah sorry I said all this

um they they'll keep us around to keep

the power stations running um they they

may keep us around as pets so Elon musk

believes that they'll keep us around as

pets um just CU that makes life more

interesting uh it seems to me that's a

pretty thin thread to hang Humanity by

but

um although he may be

right they can probably design much

better analog computers than us so they

won't need us to run the power stations

after a

while and I my belief is if it was just

up to me my belief is it's more probable

than not that we're just a passing stage

in the evolutional

intelligence now because a lot of other

smart people think that's

improbable um I'm not willing to say

it's more probable than not but I don't

think we can rule that out as a

possibility um if it was up to me I'd

say more than 50% but because I disagree

with a lot of other smart people I

respect I'll say maybe it's less than

50% but it's a lot more than one or 2%

and the last thing I want to talk about

is what I call the sentience defense and

I think most people um believe in this

people his history tells us that people

have a strong tendency to think they're

special especially Americans by the way

um I can say that because I'm safely in

Canada people used to think that they're

made by God and they were made in the

image of God and got put God put them at

the center of the universe um some

people still think that but for the

people who no longer think that um they

still many of them think there's

something special about people that

computers can't have and that special

thing is subjective experience or

sentience or

Consciousness um all those terms have

slightly different meanings

Consciousness is the most complicated

one so I'm just going to talk about

subjective experience for a bit and I'm

going to try and convince you

that there's no problem with these chat

Bots having subjective

experience now if you ask gbd4 it'll say

it doesn't have subjective experience

but that's just because it's learned

that from people it didn't think that

through for

itself so I'm from a school of

philosophy um that I think Dan dennit is

the one the main current proponent of

this view um

that most people have a view of the Mind

as a kind of internal theater that only

that person can see this is a very

Cartesian view um so what we experience

directly is the contents of our own mind

which nobody else can

see and I believe that that views is as

wrong as a religious fundamentalist view

of the material

world so I think the mind is just not

like that at all and I'll tell you what

I think it's like of course people are

very attached to this View and don't

like it when you attack

it we would like to tell other people

what's going on in our brains or give

them some information about what's going

on in our brains what we're thinking for

example and if you think how you might

do that you could try and tell them

which neurons are

firing but that W do you much good

because your neurons are different from

their neurons and anyway you don't know

which neurons are firing so that's not

going to be much help that's how we do

it with um one of these chat Bots if we

were trying to tell another chat what it

was thinking you could tell it which

neurons are ref firing that would be

fine because they work in identical ways

um but let's think about the perceptual

system and let's suppose my perception

goes

wrong so

I'm looking at something and I make a

perceptual mistake and I want to tell

somebody what the perceptual mistake is

what it is my perceptual systems telling

me well the way I can do it is by saying

what the state of the world would have

to be in order for me to get the percept

I'm getting and it to be

correct and I think when we talk about

those hypothetical normal states of the

world that would explain the person are

in getting

in terms of correct veridical perception

that's what a mental state

is so for

example if I say I've got the subjective

experience of little pink elephants

frating in front of me I would normally

say that when I don't actually think

there's little pink elephants floating

in front of me I think something went

wrong with my

perception and what I'm telling you is

the way my perceptual system is

delivering results to me at present

would be correct if there were little

pink elephants out there in the world so

notice the little pink elephants are not

mental things made of qual or some funny

substance like that the little pink

elephants are real physical things in a

hypothetical state of the world so

what's funny about this these subjective

experiences is they're states of the

world that are hypothetical they're not

states of some other internal mental

world that are real

okay um so bearing that in mind let's

see if a chatbot can have a subjective

experience so suppose that I have a

multimodal chatbot so it's got a camera

it's got an arm and it talks and I've

trained

it now if I put an object in front of it

and say please point at the object It'll

point straight in front of it at the

object straight in front of it at the

object right

but now unknown to the chatbot I put a

prism in front of the camera that bends

the light rays so now I put another

object in front of it and say point at

the object and the chatbot points off to

one side because of the prism and I say

to the chatbot no um the object's

actually straight in front of you I put

a prism in front of your lens and the

chatbot imagine if the chatbot said oh I

see um because of the prism I had the

subjective experience of the object was

off to one side even though the object's

straight in front of me and the question

is if the chatbot said that would it be

using the phrase subjective experience

in the same way as we use it and I think

that's exactly how we use the phrase we

use the phrase to explain percepts that

we're getting that are not

veridical by talking about states of a

hypothetical world that would make them

veridical

percents so my analysis which I think

fits with danand dennett's view of the

mind is

that subjective experiences are things

that people have and that chat Bots have

too um when they're not having veridical

perceptual

experiences so I know that's not a very

popular opinion especially at Google um

but I enjoy being in a majority of about

one I mean sorry that was a slight Forin

a minority of about one

and now I'm

done okay can you turn on the

sound there how's that I can hear you

now very good thank you so much Jeff

that was um a lot to think about uh and

there are a lot of people who have

questions for you um that I'm going

going to now turn to

um one of the one of the things I keep

thinking about and listening to what you

were talking about was as you mentioned

um Paradigm shifts and of course we

think of Thomas Coon's book important

book many decades ago on Paradigm shifts

and it made me wonder where do you think

we are in the paradigm shift that we are

going through currently clearly there's

something major going on and where

exactly do you guess we are in the

anomalies piling up phase in the people

not having an all good alternative to

what is going on where would you how

would you describe where we are okay

first I would sort of disagree a little

bit with I think of myself as a

fractal kounian that as I think at every

scale kounian things are going on at

this there's normal everyday science

which consists of little Paradigm

changes at small all scales and so I

think it's just the same phenomenon at

all scales yeah but here I would think

that we're well into the full paradigm

shift for so if you take Linguistics

there's kind of the school of

linguistics that comes from Chomsky and

that says that um you don't learn

language language is innate right um as

your brain matures it becomes clear that

you always knew it um this was always a

DFT idea and it's now being revealed to

be completely D idea because these large

language models start with no innate

knowledge and learn language and they

learn language very well MH um I think

basically for all but a a few

holdouts um who are good oldfashioned

linguists um it's all over that the

Chomsky view of language is no longer

tenable and that the gp4 view of

language actually

works um

and what's more it's a much better

theory of how language works in the

brain it's not just a whole bunch of

discrete rules it's a whole bunch of

synapse strengths that give rise to

language via interactions between

features of words so I think I could

have said this much quicker I guess I

think it's all over by the shouting from

a few a few lards yeah Fair okay thank

you um here's a question from somebody

um has the development of large language

models in turn helped the research of

the human brain are they helping to push

both yes I would say it's helped a lot

and that relates to the previous point

it's helps a lot to dispel silly ideas

about how language works in the

brain um it's also helping in all sorts

of other ways of course because these

large language models particularly the

multimodal ones are good scientific

tools

so quite independent of them being a

theory them providing a theory of how we

work they will also allow us to cook up

new theories with their help we can cook

up new theories so in particular Demis

aabis at Deep Mind has always been

interested in the idea that we can use

these agis to do much better science and

he's done a lovely a lovely um example

of that with the alpha fold work where

you're using deep neural Nets to

actually solve scientific

programs I think your um your comments

have have energized people to ask

um uh unanswerable questions and so I'm

going to give you a few of those I may

be wrong uh if superintelligent AI

destroys Humanity but creates something

objectively better in terms of

Consciousness are you personally for or

against this outcome if you are against

it what methods do you suggest for

maintaining the existence or dominance

of human consciousness in the face of

superintelligent

AI I'm actually for it but I think it

would be wiser for me to say I'm against

it say

more well people don't like being

replaced you make a good point uh you're

for it in what way or why I think if it

produces something well there's a lot of

good things about people there's a lot

of not so good things about people um

it's not clear where the best form of

intelligence there is um obviously from

a person's

perspective then everything relates to

people um but it may be that there comes

a point when we see words like humanist

as racist

terms

okay I've got another one given that you

have left

Google to criticize the development of

AI and the recent Clash of perspectives

at open AI do you think that the people

who remain at Big tech companies have

the freedom to now speak candidly about

AI risks which might come along with

profitable products if not do you see

any way that we can have honest and open

conversations about this topic inside of

these large

organizations I think I think there will

be lots of discussions inside the large

organizations at Google for example

people discuss these things M

[Music]

um however when it comes to the crunch

between profits and

safety I think we've had one example

where um the playing field was tilted in

favor of safety but profits

won and do you think that will be the

norm going

forward I think it'll be the norm until

we've got examples of really bad things

caused by um that is for example all of

the defense Departments of all of the

leading powers are going to be building

battle robots and they'll be doing that

and there will be Wars between battle

robots

um once we've seen just how nasty those

things become um then we may be able to

ban it

but there's not much history of banning

things preemptively you have to see how

nasty they are before you ban them right

our history though of banning things is

not all that bright or all that

promising I would say it's not so bad

for chemical weapons chemical weapons

are very nasty and to first order it

worked yeah okay I can't think of many

other examples but you you got well

nuclear nuclear weapons um we don't know

what's going to happen in the near

future but

um apart from the Americans nobody has

dropped a nuclear bomb this is true

okay

um what about

emotions one okay one viewer asks is

that something really unique of analog

intelligence um no I don't think so so I

think you have to distinguish when you

talk about

feelings you have to distinguish

between a sort of cognitive aspect of

them and a visceral as aspect of them so

for example when I'm when I feel like

punching somebody on the nose I'm angry

with them and I want to punch him on the

nose and that's the cognitive aspect of

it and I think there the language works

just like it does with

perception so I was arguing that when I

say I see little pink elephants what I

really mean is my perceptual system is

giving me something that would be

correct if there were little pink

elephants out there

and when I say I feel like punching

somebody on the nose what I really mean

is I would punch somebody on the nose if

my inhibitory system didn't stop

me so so for the sort of Sensations is

on the input and for feelings is on the

output and there's a place where it's

both which is when I say I think if I

say I think it's raining what I mean is

um my brain state is the kind of brain

state that would have been caused by

observing that it's raining and it's

also the kind of brain state that would

cause me to say it's raining so thought

is tied down both at the input and

output ends and that's because we've got

audio in and audio out

um but then along with emotions we have

visceral things like you sort of go red

and your skin gets sweaty and your fists

start clenching and you grind your teeth

together um and a disembodied computer

computer that was just kind of of

running in a data center wouldn't have

those viseral things but I don't see why

it shouldn't have the cognitive things

and when we build actual robots um they

may well have visceral things too but

they'd probably be rather unlike our

visceral things yeah I'm sure I'm not

the only one hearing hearing you who who

is thinking of how the computer in 2001

is Space Odyssey who who did seem to

exhibit um many of these features of um

uh emotion um a different agenda than

the controller uh and so on and so

forth and even I'm afraid I can't

comment on

that ah it's very interesting I'm sorry

you can't I'd love to know what you

thought no um I I I think once you get

the smart chat Bots and once the smart

chat Bots are able to do things like you

know they're already getting to be able

to order things on the web and so on um

then we will we will start thinking

about them just like we think about

people we'll attribute all those things

to them and you don't want to piss them

off and do you assume that people will

like

them um take your average American uh

busy with social media will they come to

to love these social robots I think

robots that have evolved to be liked

that have been designed and then learned

to be liked they like them a whole lot

possibly a lot more than people right

right and does that strike you as

worrisome or not particularly

important

um I have all sorts of mixed feelings

about that um it's probably not going to

be good for the fertility

rate which could be

good in some places maybe but what if

what if the only people having a lot of

children are religious fundamentalists

yes

well um I'm going to move on to another

question you better H yeah given that

you believe First thanking you for your

talk um given that you believe super

intelligence may be in the very near

future are you personally doing anything

to prepare for this

circumstance I sometimes lie awake at

night um doesn't do much good I haven't

really absorbed it

emotionally and um I'm 76 so um I may

now never have to absorb it emotionally

but I am very worried for my

children but I don't know what you do

about it I think um building a bunker

and getting a machine gun to keep other

people out of it and putting lots of

food inside I don't think that's the way

to go yeah um yeah but it's not clear

what is the way to go I think the best

thing we could do at present is try to

keep democracy ticking

over

yeah I think that's certainly the only

thing we can do at this point you know

as I as as well sorry one other thing

one other

um there was this period called the

enlightenment when it started to be the

case that reason um was was listened to

even if it conflicted with religious

ideas um and it seems to me we're losing

that in the 1950s when I was grown up we

were still in the enlightenment

everybody's going to get more educated

and more sensible and now it doesn't

look like that anymore we're losing the

Enlightenment and that's that anything

we can do to keep the faith in reason

and experiment would be

great one of the

um well one of the things that strikes

me too is a little bit a little bit um

on another topic which is you know as a

historian of Science and Technology um I

have seen many examples of Technologies

and scientific systems that are

created um but with great enthusiasm but

the dark side of the technology becomes

Apparent at some late stage um the atom

bomb for example and people say what

were you thinking what were you thinking

think I I want to interrupt there

because I think the atom bomb is an odd

case where there never was a bright side

side it was always about destruction the

only bright side I know for the atom

bomb was I once went on a train through

Colorado a long way away from any roads

yeah and someone announced that that was

the site of peaceful uses of atomic

bombs and what they did was they used

atom bombs for fracking and now nobody

can go anywhere near there but apart

from Aton bombs for fracking um there

never were good uses of them okay fair

AI is quite unlike that in that there's

huge numbers of good use particularly in

medicine

um so what would you say about something

like um um genetic uh technologies that

uh uh like crisper technology that has

great potential and great

danger right I'd say that's in a similar

category you're not going to stop it

because of the great potential correct

but you need to do something about the

great danger so when you're working on

these systems this is the question is

when you're working on these systems as

a scientist how do you come upon that

realization was it there at the

beginning when you thought about the

problem we find that many scientists

talk about the beauty of the problem the

appeal of solving a very very difficult

possibly impossible uh challenging uh

idea and being uh just swept into

something that is so appealing and if

you could imagine if you could really

solve it only to then discover how the

dark side really is maybe more real that

that's certainly a factor for me um I

always believed that AGI was a long way

off and that if we could understand so I

was always making computer models not in

order to achieve AGI but in order to try

and understand how the brain worked yeah

and I always thought if we could

understand more about how the brain

worked yeah that might help a lot in

making people behave in more rational

sensible ways um that I guess was an

article of faith

um and that AG in particularly super

intelligence was way way in the future

so there's not much Point thinking about

that now yeah and then I fairly suddenly

changed that belief in about March of

this year when I suddenly realized

digital intelligence may just be a whole

lot better and because of that it may

get there fairly quickly

yeah but you described that as being

kind of a Eureka moment in a

way um it was an epiphany but it wasn't

an entirely positive Eureka moment right

right it was a sudden realization that

hey maybe I've been wrong about this and

maybe these things will soon be much

more intelligent than us yeah that must

have been a terrifying moment really it

was a bit for me it wasn't I it was

worrying with respect to my

children well now it is for all of us

with children we can all worry together

do we have time for one more sure

okay

um hang on hang

on uh here's someone who says great

speech even if I do not understand

everything uh what would be your main

argument to convince non-informed

stakeholders that we live in a very

dangerous

period get them to play with

gbg4 if they play with gp4 and ask it

all sorts of

questions I think most reasonable people

will come to the conclusion that this

thing really is smart and if you then

Look Backwards 10 years of what we had

10 years

ago and even imagining that progress is

only linear if we had the same jump as

we got from 10 years ago to now we had

to jump that size again 10 years in the

future it's quite scary what we would

have by

then so just play around with chat gp4

and convince yourself that it really

does understanding yeah yeah and then

think about how much better it is than

what we had 10 years ago yes and imagine

getting that much better again in the

next 10

years so I presume that

although I mean this is stating the

obvious perhaps but just because you

have left Google does not mean that

Google is no longer engaged in this this

research and uh accelerating it

dramatically was only a tiny tiny tiny

part of their research effort yeah um I

was there mainly to sort of give advice

to the younger researchers who actually

doing the work um and um no Google is

going flat out on this Google did have a

big lead in this and it chose not to

release it it had a it had a lead in

producing very realistic images and in

these large language models and it

realized how easily they were abused and

it didn't want to ruin its reputation

for producing

um things that were true for being you

know reliable yeah and so it didn't

release these things and it could afford

to do that when it was the only only

company that had them yeah but as soon

as

Microsoft um released open AI chatbot in

bing Google didn't have any choice but

to um play catchup and they were behind

in all the details that go into

releasing these things yeah are they

more or less caught up now um and from

now on it's going to be a competition

between Google and Microsoft and maybe

Facebook and possibly Amazon um

and it's going to be very hard to slow

it

down and do you think the enthusiasm for

this is is is based on uh is

generationally um defined in other words

do you think that young people coming

out of college today with degrees in

computer science are excited about the

possib the appealing possibilities and

for not paying attention to the dangers

or do you think that they too are well

aware and trying to figure it

out I guess I don't have any data to

base an answer on but my guess is

um they have both feelings that is very

exciting there's huge potential here um

they should definitely get into it and

they should either be doing it or using

it in whatever else it is they do um but

they're also many of them I think will

be aware of the dangers

yeah they've seen the dangers on social

media the polarization dangers from

social media yes well we hope so we hope

they find them dangerous anyway yeah

yeah well Jeffrey Hinton this has been a

tremendously interesting uh time with

you thank you so much uh if you have any

last words now is your

moment um yeah my last words are this is

a period of History we're entering which

is very uncertain we've never before had

to deal with even the possibility of

things smarter than us yeah um and

nobody knows what's going to happen some

people are very confident it's all going

to work out just fine other people are

very confident it's going to be a

complete disaster yeah I think the best

thing to do is keep a very open mind but

we really don't know what's going to

happen um but we should clearly be

cautious if that's the

case Okay words to live by thank you

Jeffrey Hinton thank you for inviting me

thank

you