Lecture 1.2 — What are neural networks — [ Deep Learning | Geoffrey Hinton | UofT ]

Summary

本视频深入探讨了真实神经元的工作原理，这些神经元启发了人工神经网络的设计。视频中首先强调了研究神经元网络的三个主要原因：理解大脑工作方式、启发新型并行计算方式以及解决实际问题。接着，视频详细描述了单个神经元的结构，包括细胞体、轴突和树突，以及神经元之间通过突触进行通信的过程。突触通过释放神经递质来改变后神经元的去极化状态，从而实现信号传递。此外，视频还讨论了突触如何通过改变神经递质的数量或受体分子的敏感性来适应和学习。最后，视频指出大脑皮层的模块化特性，以及大脑的灵活性和通用学习算法，这些特性使得大脑能够通过经验学习新的功能。

Takeaways

• 🧠 真实神经元启发了人工神经网络的设计，它们在大脑中的网络结构是并行计算的基础。
• 🧪 通过计算机模拟帮助理解大脑，因为大脑复杂且不易直接实验。
• 💡 研究神经元网络的计算方式有助于开发更好的并行计算机，尤其是在视觉等大脑擅长的领域。
• 🚫 人工神经网络与大脑的实际工作方式可能不同，但它们解决实际问题的能力依然非常有用。
• 🌐 一个典型的皮层神经元由细胞体、轴突和树突构成，通过突触与其他神经元通信。
• ⚡️ 神经元通过在轴突上传递电位尖峰来发送消息，这些尖峰能够触发突触释放神经递质。
• 🔄 突触的适应性是学习过程的关键，通过改变突触的效能来实现。
• 🔬 突触的结构包含小囊泡，它们包含传递信号的化学物质，影响后续神经元的激活状态。
• 📉 突触的响应速度虽然慢，但它们小而节能，并且能够适应，这是它们的主要优势。
• 🧬 大脑皮层是模块化的，不同的区域负责不同的功能，这种分工是通过经验形成的。
• 🔄 大脑具有通用学习算法，能够灵活适应不同的任务，类似于FPGA（现场可编程门阵列）。
• 🧬 如果早期大脑受损，功能可以转移到其他部位，表明大脑具有可塑性，可以根据经验重新分配功能。

Q & A

为什么我们需要通过计算机模拟来理解大脑的工作方式？

大脑非常复杂且体积庞大，直接实验操作存在困难，因为大脑在被操作时可能会受损甚至死亡。计算机模拟可以帮助我们理解大脑的功能和结构，尤其是在进行实证研究时。

为什么研究大脑的并行计算对我们设计更好的并行计算机有帮助？

大脑通过大量相对较慢的神经元构成的并行网络进行计算，这种计算方式与常规串行处理器截然不同。理解这种并行计算模式有助于我们设计出更适合进行视觉等大脑擅长任务的并行计算机。

神经网络的灵感来源于哪里，它们在解决实际问题中有什么作用？

神经网络的灵感来源于大脑的工作机制。这些受大脑启发的新型学习算法在解决实际问题时非常有用，即使它们并非大脑实际工作的方式。

典型的大脑皮层神经元的物理结构包括哪些部分？

典型的大脑皮层神经元的物理结构包括细胞体、轴突（用于向其他神经元发送信息）和树突（用于接收来自其他神经元的信息）。

突触是如何工作的，它在神经元之间传递信息中扮演什么角色？

突触是神经元之间传递信息的结构。当一个神经元的轴突与另一个神经元的树突接触时，形成突触。轴突上的电位变化（称为动作电位或“spike”）会在突触处引起电荷注入，从而传递信号。

突触中的神经递质是如何影响神经元的？

突触中的神经递质通过在突触前神经元的动作电位到达时释放，然后扩散穿过突触间隙，与突触后神经元的受体分子结合。这种结合改变了受体分子的形状，从而在细胞膜上形成通道，允许特定离子流入或流出，改变突触后神经元的去极化状态。

突触如何适应并进行学习？

突触通过改变其效能来适应和学习。这可以通过改变每次动作电位到达时释放的囊泡数量，或者通过改变对释放的神经递质敏感的受体分子数量来实现。

大脑皮层的神经元数量大约有多少，每个神经元平均有多少个突触连接？

大脑皮层大约有10^11个神经元，每个神经元平均有10^4个突触连接，因此总共大约有10^15到10^14个突触权重。

大脑皮层是如何实现模块化的，模块化对学习有什么好处？

大脑皮层通过不同的部分执行不同的功能来实现模块化。这种模块化使得大脑能够更有效率地处理信息，并且具有更好的适应性，因为如果早期大脑受到损伤，功能可以重新定位到大脑的其他部分。

大脑皮层的灵活性如何体现在其学习和功能定位上？

大脑皮层的灵活性体现在它可以通过经验将通用的计算硬件转变为特定任务的专用硬件。例如，对小猫进行的实验表明，如果听觉皮层的输入被视觉输入取代，那么原本处理声音的听觉皮层可以学会处理视觉输入。

大脑的并行计算与FPGA（现场可编程门阵列）有何相似之处？

大脑的并行计算与FPGA相似，因为它们都构建了标准的并行硬件，然后通过输入信息来指定特定的并行计算任务。这种结构允许快速的并行计算，并且具有在学习新功能时的灵活性。

为什么传统的计算机需要非常快的中央处理器？

传统的计算机需要非常快的中央处理器来访问存储在程序中的线条，并执行长时间的顺序计算。这是因为它们通过存储的顺序程序来获得灵活性，这要求快速的中央处理器来访问程序中的线条并执行计算。

Outlines

00:00

🧠 大脑神经元与人工神经网络的启发

本段介绍了真实的大脑神经元如何启发了人工神经网络的设计。强调了研究神经元网络的三个主要原因：理解大脑的工作原理、探索大脑并行计算风格以提升并行计算机性能、以及开发受大脑启发的新型学习算法来解决实际问题。详细描述了神经元的物理结构，包括细胞体、轴突、树突以及突触的工作机制。突触通过改变其效能来适应，这是学习过程的关键，突触的适应性通过改变释放的囊泡数量或受体分子的敏感度来实现。

05:00

🤖 大脑的并行计算与学习算法

这段内容深入探讨了大脑的并行计算能力，以及大脑如何通过调整突触权重来学习执行各种任务，如识别物体、理解语言、制定计划和控制身体运动。描述了大脑中约有10^11个神经元，每个神经元大约有10^4个突触权重，形成了一个巨大的突触权重网络。这些权重可以在几毫秒内影响正在进行的计算，展现了比现代工作站更高的带宽到存储知识的比率。此外，还讨论了大脑皮层的模块化特性，不同的皮层区域负责不同的功能，这种模块化是后天经验形成的，而非遗传决定。通过实验，如切断幼年雪貂的听觉皮层输入并重新路由视觉输入，证明了大脑皮层具有通用学习算法，能够将通用硬件转变为针对特定任务的专用硬件。

Keywords

💡神经元

神经元是神经系统的基本工作单元，负责接收、处理和传递信息。在视频中，神经元的结构和功能是理解人工神经网络的基础，因为人工神经网络的设计灵感来源于生物神经元的工作方式。例如，视频中提到典型的皮层神经元由细胞体、轴突和树突构成，它们通过突触与其他神经元进行信息交流。

💡突触

突触是神经元之间传递信号的结构，它包含有传递化学物质的小囊泡。在视频中，突触的作用是通过释放神经递质来改变另一个神经元的电位，从而实现神经信号的传递。突触的适应性是学习过程的关键，它通过改变释放囊泡的数量或受体分子的敏感度来调整其效能。

💡神经递质

神经递质是突触中传递信号的化学物质，它们可以是兴奋性的或抑制性的。视频中提到，当一个神经冲动到达轴突时，神经递质会从囊泡中释放到突触间隙，并与后突触神经元的受体分子结合，改变细胞膜的形状，从而影响后突触神经元的去极化状态。

💡去极化

去极化是神经元细胞膜电位的变化过程，当神经元接收到足够的电荷时，会导致细胞体的某个部分（轴突结节）去极化。视频中解释说，去极化会导致神经元沿着其轴突发送一个冲动，即去极化波。这个过程是神经元传递信号的关键步骤。

💡人工神经网络

人工神经网络是受生物神经系统启发而设计的计算模型，它们模仿生物神经元的连接和处理信息的方式。视频中提到，人工神经网络可以解决实际问题，如视觉识别、语言理解等，尽管它们可能并不完全模拟大脑的工作方式。

💡并行计算

并行计算是一种计算方式，它允许多个计算任务同时进行，以提高计算效率。视频中提到，大脑通过大量并行的神经元网络进行计算，这种计算方式与常规的串行处理器不同，特别适合于大脑擅长的任务，如视觉处理。

💡

💡学习算法

学习算法是人工神经网络中用于训练模型以执行特定任务的算法。视频中指出，这些算法受到大脑的启发，可以用于解决实际问题，即使它们可能并不完全模拟大脑的工作机制。学习算法通过调整网络中的权重来优化网络的性能。

💡权重

权重是人工神经网络中连接神经元的边的属性，它们决定了信号在网络中的传递强度。在视频中，权重可以是正的或负的，并且可以通过学习过程进行调整。权重的调整是神经网络学习的基础，它们影响着网络对输入信号的响应。

💡大脑皮层

大脑皮层是大脑的外层，负责处理复杂的认知功能，如感觉知觉、语言和决策。视频中提到，大脑皮层是模块化的，不同的区域负责不同的功能，但它们看起来非常相似，表明它们可能具有通用的学习算法。

💡模块化

模块化是指系统的不同部分具有特定的功能，并且可以独立于其他部分工作。在视频中，大脑皮层的模块化意味着不同的皮层区域负责处理不同类型的信息，如视觉、听觉或语言信息。这种模块化有助于提高大脑处理信息的效率。

💡灵活性

灵活性是指系统适应新情况或学习新任务的能力。视频中提到，大脑的灵活性表现在它能够重新分配功能到不同的区域，特别是在早期损伤后，这表明大脑具有通用的学习算法，可以根据经验调整其功能。

Highlights

真实神经元为人工神经网络提供了灵感，这些网络将在本课程中学习。

研究神经元网络如何计算的三个主要原因：理解大脑工作原理、启发并行计算风格、解决实际问题。

大脑的复杂性和脆弱性使得计算机模拟成为理解大脑工作的重要工具。

大脑的并行计算风格与常规串行处理器的计算方式截然不同。

人工神经网络算法即使不完全模仿大脑工作方式，也非常有用。

皮层神经元的物理结构包括细胞体、轴突和树突。

突触是神经元间传递信息的关键结构，通过释放神经递质来实现信号传递。

神经递质通过改变膜上受体分子的形状来影响神经元的去极化状态。

突触的适应性是学习过程中最重要的特性，通过改变突触的效率实现。

大脑中有大约10^11个神经元，每个神经元平均有10^4个突触权重。

大脑的带宽到存储知识的能力远超现代工作站。

大脑皮层是模块化的，不同的皮层区域负责不同的功能。

大脑功能的定位可以通过观察血流变化来了解。

大脑皮层看起来大致相同，暗示它拥有一个灵活的通用学习算法。

大脑的灵活性允许功能在早期损伤后重新定位到其他区域。

实验表明，大脑皮层能够将通用硬件转变为针对特定任务的专用硬件。

大脑的快速并行计算与学习新功能的灵活性相结合，类似于FPGA。

传统计算机通过存储的顺序程序获得灵活性，但需要非常快的中央处理器。

Transcripts

in this video I'm going to tell you a

little bit about real neurons on the

real brain which provide the inspiration

for the artificial neural networks that

we're going to learn about in this

course in most of the course we won't

talk much about really ions but I wanted

to give you a quick overview at the

beginning there are several different

reasons to study how networks of neurons

can compute things the first is to

understand how the brain actually works

you might think we could do that just by

experiments on the brain but it's very

big and complicated and it dies when you

poke it around and so we need to use

computer simulations to help us

understand what we're discovering in

empirical studies the second is to

understand a style of parallel

computation that's inspired by the fact

that the brain can compute with a big

parallel network of relatively slow

neurons if we can understand that style

of parallel computation we might be able

to make better parallel computers it's

very different from the way computation

is done on a conventional serial

processor it should be very good for

things that brains are good at like

vision and it should also be bad for

things that brains are bad at like

multiplying two numbers together a third

reason which is the relevant one for

this course is to solve practical

problems by using novel learning

algorithms that were inspired by the

brain these algorithms can be very

useful even if they're not actually how

the brain works so most of this course

we won't talk much about how the brain

actually works it's just used as a

source of it for inspiration to tell us

that big parallel networks of neurons

can compute very complicated things I'm

going to talk more in this video though

about how the brain actually works

a typical cortical neuron has a gross

physical structure that consists of a

cell body and an axon where it sends

messages to other neurons and a

dendritic tree where it receives

messages from other neurons where an

axon from one neuron

contacts a dendritic tree of another

neuron there's a structure called the

synapse

and a spike of activity traveling along

the axon causes charge to be injected

into the postsynaptic neuron at a

synapse

a neuron generates spikes when it's

received enough charge in its dendritic

tree to depolarize a part of the cell

body called the axon hillock and when

that gets depolarized the neuron sends a

spike edge along its axon when the

spikes just a wave of depolarization

that travels along the axon synapses

themselves have interesting structure

they contain little vesicles of

transmitter chemical and when a spike

arrives in the axon it causes these

vesicles to migrate to the surface and

be released into the synaptic cleft

there's several different kinds of

transmitter chemical there's ones that

implement positive weights and ones that

implement negative weights the

transmitter molecules diffuse across the

synaptic cleft and bind to receptor

molecules in the membrane of the

postsynaptic neuron and by binding to

these big molecules in the membrane they

change their shape and that creates

holes in the membrane these holes are

life specific ions to flow in or out of

the postsynaptic neuron and that changes

their state of depolarization synapses

adapt and that's what most of learning

is changing the effectiveness of a

synapse they can adapt by varying the

number of vesicles that get released

when a spike arrives or by varying the

number of receptor molecules that are

sensitive to the released transmitter

molecules synapses are very slow

compared with computer memory but they

have a lot of advantages over the random

access memory on a computer they're very

small and very low power and they can

adapt that's the most important property

they use locally available signals to

change their strengths and that's how we

learn to perform

any computations the issue of course is

how do they decide how to change their

strengths what is the what are the rules

for how they should adapt so all on one

slide this is how the brain works each

neuron receives inputs from other

neurons a few of the neurons receive

inputs from the receptors it's a large

number of neurons but only a small

fraction of them and the neurons

communicate with each other in the

cortex by sending these spikes of

activity the effect of an input line on

the neuron is controlled by synaptic

weight which can be positive or negative

and these synaptic weights adapt and by

adapting these weights the whole network

learns to perform different kinds of

computation for example recognizing

objects understanding language making

plans controlling the movements of your

body you have about 10 to the 11 neurons

each of which has about 10 to the 4

weights so you probably have 10 to the

15 or maybe only 10 to the 14 synaptic

weights and a huge number of these

weights quite a large fraction of them

can affect the ongoing computation in a

very small fraction of a second in a few

milliseconds that's much better

bandwidth to stored knowledge than even

a modern workstation has one final point

about the brain is that the cortex is

modular or at least it learns to be

modular different bits of the cortex end

up doing different things genetically

the inputs from the senses go to

different bits of the cortex and that

determines a lot about what they end up

doing if you damage the brain of an

adult local damage to the brain causes

specific effects damage to one place

might cause you to lose your ability to

understand language damage to another

place might cause you to lose your

ability to recognize objects we know a

lot about how functions are located in

the brain because when you use a part of

the brain for doing something it

requires energy

and so it demands more blood flow and

you can see the blood flow in a brain

scanner so that allows you to see which

bits of the brain you are using for

particular tasks but the remarkable

thing about cortex is it looks pretty

much the same all over and that strongly

suggests that it's got a fairly flexible

Universal learning algorithm in it

that's also suggested by the fact that

if you damage the brain early on

functions will relocate to other parts

of the brain so it's not genetically

predetermined at least not directly

which part of the brain will perform

which function this convincing

experiments on baby ferrets that show

that if you cut off the input to the

auditory cortex it comes from the ears

and instead reroute the visual input to

auditory cortex than the auditory cortex

that was destined to deal with sounds

will actually learn to deal with visual

input and create neurons that look very

like the neurons in the visual system

this suggests the cortex is made of

general-purpose stuff that has the

ability to turn into special purpose

hardware for particular tasks in

response to experience and that gives

you a nice combination of rapid parallel

computation once you've learned plus

flexibility so you can put you can learn

new functions so you're learning to do

the parallel computation it's quite like

an FPGA where you build some standard

parallel hardware and then after it's

built you put in information that tells

it what particular parallel computation

to do conventional computers get their

flexibility by having a stored

sequential program but this requires

very fast central processors to access

the lines in the scheduled program and

perform long sequential computations