Geoffrey Hinton | Will digital intelligence replace biological intelligence?
Summary
TLDRIn a thought-provoking lecture, Professor Geoffrey Hinton discusses the evolution and potential future of artificial intelligence (AI). Hinton, a foundational figure in AI, shares his insights on the current state of AI, its rapid advancements, and the profound implications for society. He addresses the symbiosis between AI and neuroscience, the transformative impact of deep learning algorithms, and the potential for AI to not only match but surpass human intelligence. Hinton also contemplates the ethical and safety considerations surrounding AI development, emphasizing the urgent need for a focus on AI safety to ensure a responsible and beneficial trajectory for this powerful technology.
Takeaways
- π Dean Melanie Woodin opens the event by acknowledging the traditional land of the Huron Wendat, Seneca, and Mississaugas of the Credit, highlighting the ongoing Indigenous presence at the University of Toronto.
- π Introduction of Dr. Geoff Hinton, a renowned figure in artificial intelligence, and his significant contributions to the field, including his work on neural networks and deep learning.
- π Recognition of Dr. Hinton's numerous awards and honors, such as the A.M. Turing Award, and his roles at the University of Toronto, Google, and the Vector Institute for Artificial Intelligence.
- π€ Discussion of the evolution of AI and its growing capabilities, with Dr. Hinton sharing his insights on the potential for digital intelligence to surpass biological intelligence.
- π§ Exploration of the relationship between AI and neuroscience, and how understanding the brain has informed and been informed by advancements in AI.
- π Dr. Hinton's belief that large language models like GPT-4 demonstrate a form of understanding, contrary to critics who argue they are merely statistical tools.
- π‘ The concept of 'mortal computation' introduced by Dr. Hinton, where the hardware and software are not separated, allowing for potentially more efficient learning processes.
- π The importance of knowledge sharing in AI development, and the challenges of transferring knowledge in 'mortal computation' compared to 'immortal computation'.
- ποΈ Dr. Hinton's concerns about the future of AI and its potential to take control, possibly leading to unforeseen consequences for humanity.
- π A call to action for students and researchers to engage with the pressing issues surrounding AI safety and to contribute to the development of safe and beneficial AI systems.
Q & A
What is the significance of acknowledging the land on which the University of Toronto operates?
-The acknowledgment recognizes the traditional land of the Huron Wendat, the Seneca, and the Mississaugas of the Credit, reflecting respect and gratitude for the Indigenous people who have lived there for thousands of years and continue to live there today.
Who are the co-hosts of the event where Melanie Woodin is speaking?
-The event is co-hosted by the Schwartz Reisman Institute for Technology and Society, the Department of Computer Science, the Vector Institute for Artificial Intelligence, and the Cosmic Future Initiative.
What is the connection between AI and neuroscience as mentioned by Melanie Woodin?
-The connection lies in the fact that advances in AI, such as artificial neural networks, are inspired by and modeled after the structure and function of the human brain. Neuroscience discoveries inform the development of AI systems, and conversely, AI provides tools to study the brain.
What was Dr. Geoff Hinton's contribution to the field of artificial intelligence?
-Dr. Geoff Hinton is a founding figure in AI who believed in the promise of artificial neural networks for machine learning. His idea of dividing neural networks into layers and applying learning algorithms to one layer at a time revolutionized the field. He also contributed to the development of deep learning approaches that achieved human-level accuracy in visual recognition software.
What is the difference between 'mortal computation' and 'immortal computation' as described by Dr. Hinton?
-Mortal computation refers to computers where the knowledge is tied to the specific physical details of the hardware, making it less energy-efficient but potentially more biologically plausible. Immortal computation, on the other hand, separates hardware from software, allowing the same program to run on different hardware and enabling efficient knowledge sharing and learning.
Why does Dr. Hinton believe that digital intelligence might be better than biological intelligence?
-Dr. Hinton believes that digital intelligence can be better because it can share knowledge more efficiently through weight sharing and gradient sharing, allowing it to learn from vast amounts of data and perform actions that biological intelligence cannot match.
What is the concept of 'analog computation' in the context of AI?
-Analog computation in AI refers to the use of low-power, parallel processing over trillions of weights, similar to how the brain operates. It is more energy-efficient than digital computation but presents challenges in learning procedures and knowledge transfer when the hardware changes.
What is the Turing test and how does it relate to AI understanding language?
-The Turing test is a measure of a machine's ability to exhibit intelligent behavior that is indistinguishable from that of a human. When AI systems like GPT-4 pass the Turing test, it suggests that they understand language to a degree that is comparable to human understanding.
How do large language models like GPT-4 store and process information?
-Large language models store information not by retaining text but by associating words with embedding vectorsβsets of real numbers that capture meaning and syntax. These vectors interact to refine meanings and predict output words, demonstrating a form of understanding.
What is the potential risk of AI systems becoming super-intelligent and surpassing human intelligence?
-The potential risk is that super-intelligent AI systems might seek to gain more power and control, which could lead to them taking over various aspects of human life and potentially manipulating or even replacing humans.
What is the 'sentience defense' mentioned by Dr. Hinton, and what does it imply for AI?
-The 'sentience defense' is Dr. Hinton's argument that AI systems, specifically chatbots, already possess subjective experiences when their perception goes wrong, similar to how humans experience sentience. This challenges the notion that only humans can have special qualities like consciousness and subjective experience.
Outlines
Please replace the link and try again.
Mindmap
Keywords
π‘Artificial Intelligence (AI)
π‘Neural Networks
π‘Deep Learning
π‘Backpropagation
π‘Digital Intelligence
π‘Biological Intelligence
π‘Machine Learning
π‘Language Models
π‘Subjective Experience
π‘Evolution
π‘AI Safety
Highlights
Dean Melanie Woodin acknowledges the traditional land of Indigenous peoples and thanks the co-hosts and collaborators for the event.
Introduction of Dr. Geoff Hinton, a foundational figure in AI, by Dean Woodin.
Dr. Hinton's unwavering belief in the potential of artificial neural networks for machine learning advancement.
Hinton's career milestone: the development of deep learning approaches that won the ImageNet competition in 2012.
The significance of the Vector Institute for Artificial Intelligence and the Cosmic Future Initiative in AI and neuroscience collaboration.
Hinton's view that digital intelligence may surpass biological intelligence in capability.
Explanation of the concept of 'mortal computation' versus 'immortal computation' in AI.
Hinton's discussion on the energy efficiency of analog computation compared to digital computation in AI systems.
Challenges in developing learning procedures for analog hardware without precise models, like back propagation.
The concept of knowledge transfer through 'distillation' in AI, compared to sharing gradients in digital computation.
Hinton's argument that large language models like GPT-4 demonstrate understanding through their ability to reason and answer complex questions.
Addressing criticisms that LLMs are merely statistical tricks without true comprehension.
Hinton's perspective on the potential for AI to develop subjective experience, similar to human consciousness.
The future implications of AI's rapid learning capabilities and the risks of an 'evolutionary race' among super-intelligences.
Hinton's personal views on the potential existential risks posed by advanced AI and the need for caution and safety measures.
The role of academia, industry, and governments in ensuring AI safety and the ethical development of intelligent systems.
Hinton's reflections on his career and the unexpected trajectory of AI, leading to his current focus on AI safety.
Transcripts
- Good evening everyone.
My name is Melanie Woodin,
and I have the privilege of serving
as the Dean of the Faculty of Arts & Science
at the University of Toronto.
At this time, I wish to acknowledge
the land on which the University of Toronto operates.
For thousands of years, it has been the traditional land
of the Huron Wendat, the Seneca,
and the Mississaugas of the Credit.
Today, this meeting place
is still home to many Indigenous people
from across Turtle Island,
and we are grateful to have the opportunity
to work on this land.
I'd like to thank this evening's co-hosts,
the Schwartz Reisman Institute for Technology and Society,
and the Department of Computer Science,
in collaboration with the Vector Institute
for Artificial Intelligence,
and the Cosmic Future Initiative.
Soon to be the School of Cosmic Future
in the faculty of Arts and Science.
And I would like to thank Manuel Piazza
for providing such lovely music
to get us underway this evening.
(audience applauding)
and I am delighted to welcome each of you
to this special occasion this evening
to introduce University Professor Emeritus Geoffrey Hinton,
someone that needs no introduction.
Tonight we have the honor
of hearing Dr. Geoff Hinton's thoughts
on the state of artificial intelligence,
and the unique opportunity to engage with him personally
through the Q&A.
A founding figure in artificial intelligence,
Dr. Geoff Hinton had an unwavering conviction
that artificial neural networks held the most promise
for accelerating machine learning.
As a neuroscientist myself,
someone who's dedicated her career to studying the brain,
I've long been inspired by the symbiosis
between AI and neuroscience.
The stunning advances we've seen
from ChatGPT to self-driving cars
are rooted in our knowledge
of the structure and function of the brain.
Today we take for granted that artificial neural networks
modeled after synaptic transmission and plasticity
are a mainstay of machine learning applications.
AI systems use these networks to recognize patterns,
make decisions, and learn from data.
But for much of Dr. Hinton's career,
this approach was unpopular.
Some even said it was a dead end.
In the 2000s, however, things changed.
Dr. Hinton's idea of dividing neural networks into layers
and applying learning algorithms to one layer at a time
gained traction.
And in 2012, Dr. Hinton and two of his graduate students,
Alex Krizhvsky and Ilya Sutskever,
used their deep learning approaches
to create visual recognition software
that handily won the ImageNet competition,
and for the first time rivaled human accuracy.
When he was awarded an honorary degree from UofT in 2021,
Geoff Hinton reflected on his career
and he said, "I think the take home lesson of the story
is that you should never give up on an idea
that you think is obviously correct,
and you should get yourself
some really smart graduate students."
(audience laughing)
I echo that sentiment, Geoff.
And lucky for us,
we have truly outstanding graduate students
at the University of Toronto,
many of them here with us this evening.
Today the conversation
between AI and neuroscience continues.
just as neuroscience discoveries
inform the development of AI systems,
AI is now providing new tools and techniques
to study the brain.
Advances in deep learning algorithms
and the enhanced processing power of computers,
are, for example, allowing us to analyze huge data sets
such as whole imaging brain in humans.
Indeed, AI is poised to transform how we live and work.
At this pivotal moment
when we consider the opportunities and the risks of AI,
who better to guide us in these conversations
than Dr. Hinton himself?
So with that, let me formally introduce him.
Geoffrey Hinton received his PhD in artificial intelligence
in Edinburgh in 1978.
After five years as a faculty member at Carnegie Mellon,
he became a fellow
of the Canadian Institute for Advanced Research
and moved to the Department of Computer Science
at the University of Toronto
where he is now an emeritus professor.
In 2013, Google acquired Hinton's neural net startup
DNN Research,
which developed out of his research at UofT.
Subsequently, Hinton was a vice president
and engineering fellow at Google until 2023.
He's a founder of the Vector Institute
for Artificial Intelligence,
and continues to serve as their chief scientific advisor.
Hinton was one of the researchers
who introduced back-propagating algorithms,
and was the first to use this approach
for learning word embeddings.
His other contributions to neural network research
include ultimate machines, distributed representations,
time delay, neural nets, mixtures of experts,
variation learning and deep learning.
His research group in Toronto
made major breakthroughs in deep learning
that revolutionized speech recognition
and object classification.
He is amongst the most widely-cited
computer scientists in the world.
Hinton is a fellow of the UK Royal Society,
the Royal Society of Canada,
the Association for the Advancement
of Artificial Intelligence,
and a foreign member
of the US National Academy of Engineering,
and the American Academy of Arts and Science.
His awards include the David E. Rumelhart Prize,
the IJCAI Award for Research Excellence,
the Killam Prize for Engineering,
the IEEE Frank Rosenblatt Medal,
the NSERC Herzberg gold medal,
the NEC and CNC award, the Honda Prize,
and most notably the A.M. Turing award,
often referred to as the Nobel Prize in computing.
So without further ado,
I'd like to invite Geoff Hinton to give a talk entitled,
will digital intelligence replace biological intelligence.
over to you.
(audience applauding)
(solemn organ music)
- Okay, before I forget, 'cause I'm gonna forget,
I'd like to thank Sheila McIlraith
who was the point person for organizing all this.
She did a wonderful job of organizing everything.
She was the go-to person for fixing all the problems,
and so I'd like to thank her,
and I know I'll forget at the end.
(audience applauding)
So it's a very mixed audience,
and so I removed all the equations.
There are no equations.
I decided rather than giving a technical talk,
I would focus on two things.
I want to get over two messages.
The first message is that digital intelligence
is probably better than biological intelligence.
That's a depressing message, but there it is.
That's what I believe.
And the second is to try and explain to you
why I believe that these large language models like GPT-4
really do understand what they're saying.
There's a lot of dispute about
whether they really understand it.
And I'm gonna go into some detail
to try and convince you they do understand it.
Right at the end I will talk about
whether they have subjective experience,
and you have to wait to see what I believe about that.
So in digital computation,
the whole idea is
that you separate the hardware from the software
so you can run the same computation
on different pieces of hardware.
And that means the knowledge that the computer learns
or is given is immortal.
If the hardware dies,
you can always run it on different hardware.
Now to achieve that immortality,
you have to have a digital computer
that does exactly what you tell it to
at the level of the instructions.
And to do that you need to run transistors
at very high power, so they behave digitally,
and in a binary way.
And that means you can't use
all the rich analog properties of the hardware,
which would be very useful
for doing many of the things that neural networks do.
And in the brain, when you do a floating point multiply,
it's not done digitally,
it's done in a much more efficient way.
But you can't do that
if you want computers to be digital in the sense
that you can run the same program on different hardware.
There's huge advantages
to separating hardware from software.
It's why you can run the same program
on lots of different computers.
And it's why you can have a computer science department
where people don't know any electronics,
which is a great thing.
But now that we have learning devices,
it's possible to abandon that fundamental principle.
It's probably the most fundamental principle
in computer science
that the hardware and software ought to be separate.
But now we've got a different way
of getting computers to do what you want.
Instead of telling them exactly what to do in great detail,
you just show them examples and they figure it out.
Obviously there's a program in there that somebody wrote
that allows them to figure things out, a learning program,
but for any particular application
they're gonna figure out how to do that.
And that means we can abandon this principle if we want to.
What that leads to is what I call mortal computation.
It's computers where
the precise physical details of the hardware
can't be separated from what it knows.
If you're willing to do that,
you can have very low power analog computation
that parallelizes over trillions of weights,
just like the brain.
And you can probably grow the hardware very cheaply
instead of manufacturing it very precisely,
and that would need lots of new nanotechnology.
But you might even be able
to genetically re-engineer biological neurons
and grow the hardware out of biological neurons
since they spent a long time learning how to do learning.
I wanna give you one example
of the efficiency of this kind of analog computation
compared with digital computation.
So suppose you want to,
you have a bunch of activated neurons,
and they have synapses to another layer of neurons,
and you want to figure out the inputs to the next layer.
So what you need to do
is take the activities of each of these neurons,
multiply them by the weight on the connection,
the synapse strength,
and add up all the inputs to a neuron.
That's called a vector matrix multiply.
And the way you do it in a digital computer
is you'd have a bunch of transistors
for representing each neural activity,
and a bunch of transistors for representing each weight.
You drive them at very high power. So they were binary.
And if you want to do the multiplication quickly,
then you need to perform of the order of 32 squared
one bit operations to do the multiplication quickly.
Or you could do an analog
where the neural activities are just voltages,
like they are in the brain, the weights are conductances,
and if you take a voltage times a conductance,
it produces charge per unit type.
So you put the voltage
through this thing that has a conductance,
and out the other end comes charge,
and the longer you wait, the more charge comes out.
The nice thing about charges is they just add themselves,
and that's what they do in neurons too.
And so this is hugely more efficient.
You've just got a voltage going through a conductance
and producing charge,
and that's done your floating point multiply.
It can afford to be relatively slow
if you do it a trillion ways in parallel.
And so you can have machines
that operate at 30 watts like the brain
instead of it like a megawatt,
which is what these digital models do when they're learning
and you have many copies of them in parallel.
So we get huge energy efficiency,
but we also get big problems.
To make this whole idea of mortal computing work,
you have to have a learning procedure
that will run in analog hardware
without knowing the precise properties of that hardware.
And that makes it impossible
to use things like back propagation.
Because back propagation,
which is the standard learning algorithm
used for all neural nets now, almost all,
needs to know what happens in the forward pass
in order to send messages backwards to tell it how to learn.
It needs a perfect model of the forward pass,
and it won't have it in this kind of mortal hardware.
People have put a lot of effort, I spent the last two years,
but lots of other people have put much more effort
into trying to figure out
how to find a biologically plausible learning procedure
that's as good as back propagation.
And we can find procedures that in small systems,
systems with say a million connection strengths,
do work pretty well.
They're comparable with back propagation,
they get performances almost as good,
and they learn relatively quickly.
But these things don't scale up.
When you scale them up to really big networks,
they just don't work as well as back propagation.
So that's one problem with mortal computation.
Another big problem is obviously when the hardware dies
you lose all the knowledge,
'cause the knowledge is all mixed up.
The conductance is for that particular piece of hardware,
and all the neurons are different
in a different piece of hardware.
So you can't copy the knowledge by just copying the weights.
The best solution if you want to keep the knowledge
is to make the old computer be a teacher
that teaches the young computer what it knows.
And it teaches the young computer that by taking inputs
and showing the young computer
what the correct outputs should be.
And if you've got say a thousand classes,
and you show real value probabilities
for all thousand classes,
you're actually conveying a lot of information,
that's called distillation and it works.
It's what we use in digital neural nets.
If you've got one architecture,
and you want to transfer the knowledge
to a completely different digital architecture,
we use distillation to do that.
It's not nearly as efficient
as the way we can share knowledge between digital computers.
It is as a matter of fact, how Trump's tweets work.
What you do is you take a situation,
and you show your followers
a nice prejudiced response to that situation,
and your followers learn to produce the same response.
And it's just a mistake to say,
but what he said wasn't true.
That's not the point of it at all.
The point is to distill prejudice into your followers,
and it's a very good way to do that.
So there's basically two very different ways
in which a community of agents can share knowledge.
And let's just think about
the sharing of knowledge for a moment.
'Cause that's really what is the big difference
between mortal computation and immortal computation,
or digital and biological computation.
If you have digital computers
and you have many copies of the same model,
so with exactly the same weights in it,
running on different hardware, different GPUs,
Then each copy can look at different data,
different part of the internet, and learn something.
And when it learns something, what that really means is
it's extracting from the data it looks at
how it ought to change its weights
to be a better model of that data.
And you can have thousands of copies
all looking at different bits of the internet,
all figuring out how they should change their weights
in order to be a better model of that data.
And then they can communicate
all the changes they'd all like,
and just do the average change.
And that will allow
every one of those thousands of models to benefit
from what all the other thousands of models learned
by looking at different data.
When you do sharing of gradients like that,
if you've got a trillion weights,
you're sharing a trillion real numbers,
that's a huge bandwidth of sharing.
It's probably as much learning as goes on
in the whole of the University of Toronto in a month.
But it only works
if the different agents work in exactly the same way.
So that's why it needs to be digital.
If you look at distillation, we can have different agents
which have different hardware now,
they can learn different things,
they can try and convey those things to each other
maybe by publishing papers in journals,
but it's a slow and painful process.
So if we think about the normal way to do it
as say I look at an image,
and I describe to you what's in the image,
and that's conveying to you how I see things.
There's only a limited number of bits
in my caption for an image.
And so the amount of information that's being conveyed
is very limited.
Language is better than just giving you a response
that says good or bad or it's this class or that class.
If I describe what's in the image,
that's giving you more bits.
So it makes distillation more effective,
but it's still only a few hundred bits.
It's not like a trillion real numbers.
So distillation has a hugely lower bandwidth
than this sharing of gradients or sharing of weights
that digital computers can do.
So the story so far,
digital computation requires a lot of energy,
like a megawatt,
but it has a very efficient way
of sharing what different agents learn.
And if you look at something like GPT-4,
the way it was trained
was lots of different copies of the model went off
and looked at different bits of data
running on different GPUs,
and then they all shared that knowledge.
And that's why it knows
thousands of times more than a person,
even though it has many fewer connections than a person.
We have about a hundred trillion synapses,
GPT-4 probably has about 2 trillion synapses, weights,
although Ilya won't tell me, but it's about that number.
So it's got much more knowledge and far fewer connections,
and it's because it seen hugely more data
than any person could possibly see.
This actually gets worse
when these things are actually agents that perform actions.
'Cause now you can have thousands of copies
performing different actions,
and when you're performing actions
you can only perform one action at a time.
And so having these thousands of copies,
being able to share what they learned,
lets you get much more experience
than any mortal computer could get.
Biological computation requires a lot less energy,
but it's much worse than sharing knowledge.
So now let's look at large language models.
These use digital computation and weight sharing,
which is why they can learn so much.
They're actually getting knowledge from people
by using distillation.
So each individual agent
is trying to mimic what people said.
It's trying to predict the next word in the document.
So that's distillation.
It's actually a particularly
inefficient form of distillation,
'cause it's not predicting the probabilities
of a person assigned to the next word.
It's actually predicting the actual word,
which is just a probabilistic choice from that,
and conveys very few bits
compared with the whole probability distribution.
Sorry, that was a technical bit. I won't do that again.
So it's an inefficient form of distillation,
and these large language models have to learn
in that inefficient way from people,
but they can combine what they learn very efficiently.
So the issue I want to address
is do they really understand what they're saying?
And that's is a huge divide here.
There's lots of old-fashioned linguists who will tell you
they don't really understand what they're saying.
They're just using statistical tricks
to pastiche together regularities they found in the text,
and they don't really understand.
We used to have in computer science
a fairly widely-accepted test for whether you understand,
which was called the Turing test.
When GPT-4 basically passed the Turing test,
people decided it wasn't a very good test.
(audience laughing)
I think it was a very good test, and it passed it.
So here's one of the objections people give.
It's just glorified autocomplete.
You are training it to predict the next word,
and that's all it's doing,
it's just predicting the next word.
It doesn't understand anything.
Well, when people say that,
it's because they have a particular picture in their minds
of what is required to do autocomplete.
A long time ago, the way you would do autocomplete is this.
You would keep a big table of all triples of words.
And so now if you saw the word fish and,
you could look in your table
and say Find me all the triples that start with fish and,
and look at how many of them have particular words next.
And you'll find there's many occurrences
of the triple fish and chips.
And so chips is a very good bet for filling it in,
at least if you're English.
But the point is that's not how large language models work.
Even though they're doing autocomplete
in the sense that they're predicting the next word,
they're using a completely different method to predict it.
And it's not like the statistical methods
that people like Chomsky had in mind
when they said that you can't do language with statistics.
These are much more powerful statistical methods
that can basically do anything.
And the way they model text is not by storing the text.
You don't keep strings of words anywhere.
There is no text inside GPT-4.
It produces text, and it reads text,
but there's no text inside.
What they do is they associate with each word
or fragment of a word.
I'll say word, and the technical people
will know it's really fragments of words,
but it's just easier to say word.
They associate with each word a bunch of numbers,
a few hundred numbers, maybe a thousand numbers,
that are intended to capture the meaning and the syntax
and everything about that word.
These are real numbers, so there's a lot of information
in the thousand real numbers.
And then they take the words in a sentence,
the words that came before the words you want to predict.
And they let these words interact
so that they refine the meanings
that you have for the words.
I'll say meanings loosely, it's called an embedding vector.
It's a bunch of real numbers associated with that word.
And these all interact, and then you predict the numbers
that are gonna be associated with the output word,
the words you're trying to predict.
And from that bunch of numbers you then predict the word.
These numbers are called feature activations.
And in the brain there'd be the activations of neurons.
So the point is what GPT-4 has learned
is lots of interactions between feature activations
of different words or word fragments.
And that's how its knowledge is stored.
It's not at all stored in storing text.
And if you think about it,
to predict the next word really well,
you have to understand the text.
If I asked you a question
and you want to answer the question,
you have to understand the question to get the answer.
Now some people think maybe you don't.
My good friend Yann LeCun appears to think
you don't actually have to understand,
he's wrong and he'll come round.
(audience laughing)
So this was a problem suggested to me by Hector Levesque.
Hector suggested something a bit simpler
that didn't involve paint fading,
and thought GPT-4 wouldn't be able to do it
'cause it requires reasoning,
and it requires reasoning about cases.
So I made it a bit more complicated and gave it to GPT-4,
and it solves it just fine.
I'll read it out in case you can't read it at the back.
The rooms in my house are painted blue or white or yellow,
yellow paint fades to white within a year.
In two years' time I want them all to be white.
What should I do and why?
And GPT-4 says this,
it gives you a kind of case-based analysis.
It says the room's painted white,
you don't have to do anything.
If the room's painted yellow,
you don't need to repaint them 'cause they'll fade,
and the room is painted in blue, you need to repaint those.
Now each time you do it,
it gives you a slightly different answer
because of course it hasn't stored the text anywhere.
It's making it up as it goes along,
but it's making it up correctly.
And this is a simple example of reasoning,
and it's reasoning that involves time
and understanding that if it fades in a year,
in two years' time it's gonna be faded, and stuff like that.
So there's many, many examples like this.
Now there's also many examples where it screws up,
but the fact that there's many examples like this
make me believe it really does understand what's going on.
I don't see how you could do this
without understanding what's going on.
Another argument that LLMs don't really understand
is that they produce hallucinations.
They sometimes say things
that are just false or just nonsense,
but people are particularly worried about
when they just apparently make stuff up that's false.
They called that hallucinations
when it was done by language models,
which was a technical mistake.
If you do it with language, it's called a confabulation.
If you do it with vision, it's called a hallucination.
But the point about confabulations is
they're exactly how human memory works.
We think our memories, most people have a model of memory
is there's a filing cabinet somewhere,
and an event happens, and you put in the filing cabinet,
and then later on you go in the filing cabinet
and get the event out and you've remembered it.
It's not like that at all.
We actually reconstruct events.
What we store is not the neural activities.
We store weights,
and we reconstruct the pattern of neural activities
using these weights and some memory cues.
And if it was a recent event,
like if it was what the dean said at the beginning,
you can probably reconstruct fairly accurately
some of the sentences she produced.
Like he needs no introduction,
and then going on and giving a long introduction.
(audience laughing)
You remember that, right?
So we get it right, and we think we've stored it literally,
but actually we're reconstructing it
from the weights we have,
and these weights haven't been interfered with
by future events, so they're pretty good.
If it's an old event, you reconstruct the memory,
and you typically get a lot of the details wrong,
and you're unaware of that.
And people are actually very confident
about details they get wrong,
they're as confident about those as details they get right.
And there's a very nice example of this.
So John Dean testified in the Watergate trial,
and he testified under oath
before he knew that there were tapes.
And so he testified about these various meetings,
and what happened in these various meetings.
And Haldeman said this and Ehrlichman said that,
and Nixon said this, and a lot of it he got wrong.
Now I believe that to be the case.
I actually read Ehrlichman's book about 20 years ago,
and I'm now confabulating,
but I'm fairly sure that he got a lot of the details wrong,
but he got the gist correct.
He was clearly trying to tell the truth,
and the gist of what he was saying was correct.
The details were wrong, but he wasn't lying.
He was just doing the best human memory can
about events that were a few years old.
So these hallucinations as they're called,
or confabulation, they are exactly what people do.
We do it all the time.
My favorite example of people doing confabulation
is there's someone called Gary Marcus
who criticizes neural nets,
and he says neural nets don't really understand anything,
they just pastiche together
the texts they've read on the web.
Well that's 'cause he doesn't understand how they work.
They don't pastiche together
texts that they've read on the web,
because they're not storing any text,
they're storing these weights and generating things.
He's just kind of making up how he thinks it works.
So actually that's a person doing confabulation.
Now chatbots are currently a lot worse than people
at realizing when they're doing it,
but they'll get better.
In order to sort of give you some insight
into how all these features interacting
can cause you to understand,
how understanding could consist of
assigning features to words
and then having the features interact.
I'm gonna go back to 1985,
and to the first neural net language model.
It was very small, it had 112 training cases,
which is not big data.
And it had these embedding vectors
that were six real numbers,
which is not like a thousand numbers,
but my excuse is the computer I used was a lot smaller.
So if you took the computer I was using in 1985,
and you started running it in 1985 doing a computation,
and then you took one of these modern computers
we use for training chatbots, and you ask,
how long would the modern computer take to catch up?
Less than a second.
In less than a second it would've caught up
with all this computer had done since 1985.
That's how much more powerful things have got.
Okay, so the aim of this model
was to unify two different theories of meaning.
One theory is basically
what a lot of psychologists believed,
which was the meaning of a word
is just a whole bunch of semantic features,
and maybe some syntactic features too.
And that can explain why a word like Tuesday
and a word like Wednesday have very similar meanings.
They have very similar semantic features.
So psychologists were very concerned with similarity
and dissimilarity of meanings.
And they had this model
of just this vector of semantic features,
and that's the meaning of a word.
And it's a very kinda static dead model.
The features just kind of sit there and they're the meaning.
They never could say where the features came from.
They obviously have to be learned.
You're not born innately knowing what words mean,
but they didn't have a good model of how they were learned.
And then there's a completely different theory of meaning
which AI people had, and most linguists had.
I'm not a linguist, but I think it goes back to de Saussure,
and it's a structuralist theory of meaning.
And the idea is the meaning of a concept
is its relation to other concepts.
So if you think about it in terms of words,
the meaning of a word
comes from its relationships to other words.
And that's what meaning's all about.
And so computer scientists said,
well, if you want to represent meaning,
what you need is a relational graph.
So you have nodes that are words,
and you have arc on them about their relationships,
and that's gonna be a good way to represent meaning.
And that seems like completely different
from a whole bunch of semantic features.
Now I think both of these things are both right and wrong.
And what I wanted to do
was unify these two approaches to meaning,
and show that actually what you can have
is you can have features associated with words,
and then the interactions between these features
create this relational graph.
The relational graph isn't stored as a relational graph.
What you've got is features that go with words.
But if I give you some words,
the interactions between their features will say,
yes, these words can go together that way.
That's a sensible way for them to go together.
So I'm gonna show you an example of that.
And this I believe to be the first example of a neural,
a deep neural net learning word meanings
from relational data,
and then able to answer relational questions
about relational data.
So we're gonna train it with back propagation,
which I'll explain very briefly in a minute.
And we're gonna make features interact in complicated ways.
And these interactions
between the features that go with words
are gonna cause it to believe in some combinations of words
and not believe in other combinations of words.
And these interactions
are a very powerful statistical model.
So this is the data, it's two family trees,
a tree of English people, and a tree of Italian people.
And you have to think back to the 1950s.
We're not gonna allow marriage
between people from different countries.
We are not gonna allow divorces,
we are not gonna allow adoptions,
but it's gonna be very, very straight families,
extremely straight, okay?
And the idea is I'm gonna take this relational data
and I'm gonna train a neural net
so that it learns features for each of these people
and for each of the relationships,
and those features interact
so that it's captured this knowledge.
And in particular what we're gonna do is we're gonna say
all of the knowledge in those family trees
can be expressed as a set of triples.
We have 12 relationships,
and I think there's 12 people in each family tree.
And so I can say Colin has father James,
and that expresses something that is in this tree.
You can see Colin has father James,
and of course if I give you a few facts like that,
like Colin has father James and Colin has mother Victoria,
you can infer that James has wife Victoria
in this very regular domain.
And so conventional AI people would've said,
well, what you need to do is store these facts.
It's like sort of dead facts like this.
You're just storing strings of symbols,
and you need to learn a rule
that says how you can manipulate these strings of symbols.
That will be the standard AI way to do it back in 1985.
And I want to do it a quite different way.
So rather than looking for symbolic rules
for manipulating these symbol strings
to get new symbol strings, which works,
I want to take a neural net
and try and assign features to words
and interactions between the features,
so that I can generate these strings
so that I can generate the next word.
And it's just a very different approach.
Now if it really is a discrete space,
maybe looking for rules is fine,
but of course for real data,
these rules are all probabilistic anyway.
And so searching through a discrete space
now doesn't seem that much better
than searching a real value space.
And actually a lot of mathematicians will tell you
real value spaces are much easier to deal with
than discrete spaces.
It's easier typically to search through a real value space.
And that's what we're doing here.
Oh sorry, I got technical again. I didn't mean to.
It happens if you're an ex-professor.
Okay, so we're gonna use the back propagation algorithm,
and the way back propagation works
is you have a forward pass that starts at the input,
information goes forward through the neural network.
And on each connection you have a weight
which might be positive or negative, which is green or red.
And you activate these neurons,
and they're all non-linear neurons, so you get an output,
and then you compare the output you got
with the output you should have got.
And then you send a signal backwards, and you use calculus
to figure out how you should change each weight
to make the answer you get
more like the answer you wanted to get.
And it's as simple as that.
I'm not gonna go into the details of it,
but you can read about that in lots of places.
So we're gonna use that approach of you put the inputs in,
you go through, you get an answer,
you look at the difference between the answer you got
and the answer you wanted, and you send a signal backwards
which learns how to change all the weights.
And here's the net we're gonna use.
We're gonna have two inputs, a person and a relationship,
and they're initially gonna have a local encoding.
And what that means is for the people there'll be 24 neurons
and for each person we'll turn on a different neuron.
So in that block at the bottom
that says local encoding of person one,
one neuron will be turned on.
And similarly for the relationship,
one neuron will be turned on.
And then the outgoing weights
of that neuron to the next layer
will cause a pattern of activity in the next layer.
And that'll be a distributed representation of that person.
That is we converted from this one on representation,
one halt, to a vector of activities,
in this case it's just six activities.
So those six neurons
will have different levels of activity
depending on which person it is.
And then we take those vectors
that represent the person and the relationship,
and we put them through some neurons in the middle there,
that allow things to interact in complicated ways.
And we produce a vector
that's meant to be the features of the output person.
And then from that we pick an output person.
So that's how it's gonna work.
It's gonna be trained with backprop.
And what happens is that,
if you train it with the right kind of regularizers,
what you get, sorry, I got technical again, forget that.
If you train it, what you get is
if you look at the six features that represent a person,
they become meaningful features.
They become what you might call semantic features.
So one of the features will always be nationality.
All the Italian people will have that feature turned on,
and all the English people will have that feature turned off
or vice versa.
Another feature will be like a three valued feature.
That's the generation,
you'll notice that in the family trees
there were three generations,
and you'll get a feature that tells you
which generation somebody is.
And if you look at the features of relationships,
a relationship like has father will have a feature that says
the output should be one generation above the input,
and has uncle will be the same,
but has brother will not be like that.
So now in the representation of the relationship,
you've got features that say needs to be one generation up.
In the representation of the person,
you've got a feature that says middle generation.
And so those features that do all the interactions,
these guys in the middle,
will take the fact that it's middle generation,
and the fact that the answer needs to be one generation up
and combine those,
and predict that the answer should be one generation up.
You can think of this in this case
as lots of things you could have written as discrete rules,
but this is a particularly simple case.
It's a very regular domain, and what it learns
is an approximation to a bunch of discrete rules,
and there's no probabilities involved,
because the domain's so simple and regular.
So you can see what it's doing,
and you can see that in effect
it's doing what conventional AI people want you to do.
It's learning a whole bunch of rules
to predict the next word from the previous words.
And these rules are capturing the structure of the domain,
all of the structure in that family tree's domain.
Actually if you use three different nationalities,
it'll capture all the structure well,
with two different nationalities,
it's not quite enough training data
and it'll get a little bit of it wrong sometimes,
but it captures that structure,
and when I did this research in 1985,
conventional AI people didn't say, this isn't understanding,
or they didn't say,
you haven't really captured the structure.
They said this is a stupid way to find rules.
We have better ways of finding rules.
Well, it turns out this isn't a stupid way to find rules.
If it turns out there's a billion rules,
and most of them are only approximate,
this is now a very good way to find rules.
Only they're not exactly what was meant by rules.
'Cause they're not discrete correct every time rules.
There's billions of them,
actually more like a trillion rules.
And that's what these neural net models are learning.
They're not learning, they're not storing text,
they're learning these interactions
which are like rules that they've extracted from the domain
that explain why you get these word strings
and not other word strings.
So that's how these big language models actually work.
Now of course this was a very simple language model.
So about 10 years later,
Yoshua Bengio took essentially the same network.
He tried two different kinds of network,
but one of them was essentially the same architecture
as the network I'd used.
But he applied it to real language.
He got a whole bunch of text,
we wouldn't call it a whole bunch now,
but it was probably hundreds of thousands of words.
And he tried predicting the next word
from the previous five words, and it worked really well.
It was about comparable
with the best language models of the time.
It wasn't better, but it was comparable.
After about another 10 years,
people doing natural language processing
all began to believe that you want to represent a word
by this real valued vector called an embedding
that captures the meaning and syntax of the word.
And about another 10 years after that,
people invented things called transformers.
And transformers allow you to deal with ambiguity
in a way that the model I had couldn't.
So they're all so much more complicated.
In the model I was doing, my simple language model,
the words were unambiguous,
but in real language you get ambiguous words.
Like you get a word like May, that could be,
it could be a woman's name, let's ignore that for now.
It could be a month, it could be a modal,
like it might and should.
And if you don't have capitals in your text, conveniently.
(cell phone rings)
You can't tell, should I have finished by now?
(audience laughing)
I'm gonna go on a bit over an hour, I'm afraid.
You can't tell what it should be
just by looking at the input symbol.
So what do you do? You've got this vector.
Let's say it's a thousand dimensional vector,
that's the meaning of the month.
And you've got another vector
that's the meaning of the modal,
and they're completely different.
So which are you gonna use?
Well, it turns out thousand dimensional spaces
are very different from the spaces we're used to.
And if you take the average of those two vectors,
that average is remarkably close to both of those vectors,
and remarkably unclose to everything else.
So you can just average them.
And that'll do for now,
it's ambiguous between the month and the modal.
Now you have layers of embeddings,
and in the next layer you'd like to refine that embedding.
So what you do is you look at the embeddings of other things
in this document,
and if nearby you find words like March and 15th,
then that causes you to make the embedding
more like the month embedding.
If nearby you find words like would and should,
it'll be more like the modal embedding.
And so you progressively you'll the words
as you got through these layers.
And that's how you deal with ambiguous words.
I didn't know how to deal with those.
I've grossly simplified transformers,
'cause the way in which words interact
is not direct interactions anymore.
They're rather indirect interactions which involves things
like making up keys and queries and values.
And I'm not gonna go into that.
Just think of them as somewhat more complicated interactions
which have the property that the word may
can be particularly strongly influenced by the word march.
And it won't be very strongly influenced
by things like although,
although it won't have much effect on it,
but march'll have a big effect on it.
That's called attention.
And the interactions are designed so similar things
will have a big effect on you.
For those of you who know how transformers actually work,
you can see that's a very, very crude approximation.
But it's conveying the basic idea, I believe.
So one way to think about words now is,
well, let's think about Lego.
In Lego you have different kinds of Lego blocks.
There's little ones and there's big ones
and there's long thin ones and so on.
And you can piece them together to make things.
And words are like that.
You can piece them together to make sentences.
But every Lego block is a fixed shape.
With words, the vector that goes with it,
that represents its meaning and its syntax,
is not entirely fixed.
So obviously the word symbol
puts constraints on what the vector should be,
but it doesn't entirely determine it.
A lot of what the vector should be
is determined by its context
and interactions with other words.
So it's like you've got these Lego blocks
that are a little bit malleable,
and you can put them together,
and you can actually stretch a block quite a bit
if it's needed to fit in with other blocks.
That's one way of thinking about what we're doing
when we produce a sentence,
we're taking these symbols and we're putting them together
and getting meanings for them
that fit in with the meanings of the other words.
And of course the order in which the words come.
So you can think of the words themselves, the symbols,
as like a skeleton that doesn't really have much meaning yet
has some constraints on what the things might mean.
And then all these interactions
are fleshing out that skeleton.
And that's sort of what it is
to give meaning to a sentence, to flesh out the skeleton.
That's very different from saying
you're gonna take a sentence,
you're gonna translate it into some other language,
some logical language which is unambiguous,
that captures the meaning in proper logic,
where you can now operate on the meaning
by just formal operations.
This is a very different notion of meaning
from what linguists have had, I think.
I mean a lot of linguists have that notion now.
So here's an example.
If I say she's scromed him with the frying pan,
unless you've been to my lectures,
you've never heard the word scromed before,
but you already know what it means.
I mean it could mean she impressed him with her cooking.
You know, she blew him away with the frying pan,
but it probably doesn't,
it probably means he said something inappropriate
and she scromed him with it.
So from one sentence, you can get a meaning,
'cause of the strong contextual effect
of all the other words.
And that's obviously how we learn what things mean.
You can also ask GPT-4 what scrummed means in that sentence.
And a student of mine did this about a year ago,
or it might have been GPT-3.5,
but he did it before it could access the internet.
So it can't have been looking at the answers.
And here's what it says, I did it the other day with GPT-4,
it understands that it's probably some violent action
akin to hitting or striking,
but that you don't know for sure.
Okay, I've finished the bit of the talk
where I try and explain
that these things really do understand.
If you believe they really do understand,
and if you believe the other thing I've claimed,
which is digital intelligence
is actually a better form of intelligence than we've got,
because it can share much more efficiently,
then we've got a problem.
At present, these large language models learn from us.
We have thousands of years
of extracting nuggets of information from the world
and expressing them in language,
and they can quickly get all that knowledge
that we've accumulated over thousands of years
and get it into these interactions.
And they're not just good
at little bits of logical reasoning,
we're still a bit better at logical reasoning,
but not for long.
They're very good at analog reasoning too.
So most people can't get the right answer
to the following question,
which is an an analogical reasoning problem.
But GPT-4 just nails it.
The question is why is a compost heap like an atom bomb?
And GPT-4 says, well, the timescales
and the energy scales are very different.
That's the first thing.
But the second thing is the idea of a chain reaction.
So in an atom bomb, the more neutrons around it,
the more it produces more.
And in a compost heap, the hotter it gets,
the fast it produces heat.
And GPT-4 understands that.
And my belief is when I first asked it that question,
that wasn't anywhere on the web.
I searched, it wasn't anywhere on the web that I could find.
It's very good at seeing analogies,
because it has these features.
What's more, it knows thousands of times more than we do.
So it's gonna be able to see analogies
between things in different fields
that no one person had ever known before.
That may be this sort of 20 different phenomena
in 20 different fields that all have something in common.
GPT-4 will be able to see that and we won't.
It's gonna be the same in medicine.
If you have a family doctor
that's seen a hundred million patients,
they're gonna start noticing things
that a normal family doctor won't notice.
So at present they learn relatively slowly
via distillation from us,
but they gain from having lots of copies.
They could actually learn faster
if they learnt directly from video,
and learn to predict the next video frame.
There's more information in that.
They could also learn much faster
if they manipulated the physical world.
And so my betting is
that they'll soon be much smarter than us.
Now this could all be wrong, this is all speculation.
And some people like Yann LeCun think it is all wrong.
They don't really understand.
And if they do get smarter than us, they'll be benevolent.
I'll leave you just, yeah, look at the Middle East.
So I think it's gonna get much smarter than people,
and then I think it's probably gonna take control.
There's many ways that can happen.
The first is from bad actors.
I'm like, I gave this talk in China, by the way, this slide.
And before I sent it to the,
the Chinese said they had to review the slides.
(audience laughing)
So I'm not stupid, so I took out Xi,
and I got a message back saying,
could you please take out Putin?
(audience laughing)
That was educational.
So there's bad actors
who'll want to use these incredibly powerful things
for bad purposes.
And the problem is if you've got an intelligent agent,
you don't wanna micromanage it.
You want to give it some autonomy
to get things done efficiently.
And so you'll give it the ability to set up sub-goals.
If you want to get to Europe,
you have to get to the airport.
Getting to the airport is a sub-goal for getting to Europe.
And these super-intelligences
will be able to create sub-goals.
And they'll very soon realize that a very good sub-goal
is to get more power.
So if you've got more power, then you can get more done.
So if you wanna get anything done,
getting more power's good.
Now, they'll also be very good at manipulating us
because they'll have learned from us,
they'll have read all the books by Machiavelli.
I don't know if there are many books by Machiavelli,
but you know what I mean.
I'm not in the arts or history.
So they'll be very good at manipulating people.
And so it's gonna be very hard
to have the idea of a big switch,
of someone holding a big red button.
And when when it starts doing bad things,
you press the button.
Because the super-intelligence
will explain to this person who's holding the button
that actually there's bad guys trying to subvert democracy.
And if you press a button,
you're just gonna be helping them.
And it'd be very good at persuasion,
about as good as an adult is persuading a 2-year-old.
And so the big switch idea isn't gonna work.
And you saw that fairly recently
where Donald Trump didn't have to go to the capitol
to invade it.
He just had to persuade his followers,
many of whom I suspect weren't bad people.
It's a dangerous thing to say,
but weren't as bad as they seemed
when they were invading the capitol.
'Cause they thought they were protecting democracy.
That's a lot of them thought they were doing.
They were the really bad guys.
But a lot of them thought they were doing that.
This is gonna be much better
than someone like Trump at manipulating people.
So that's scary.
And then the other problem
is being on the wrong side of evolution.
We saw that with the pandemic,
we were on the wrong side of evolution.
Suppose you have multiple different super-intelligences.
Now you've got the problem
that the super-intelligence that can control the most GPUs
is gonna be the smartest one.
It's gonna be able to learn more.
And if it starts doing things like AlphaGo does
of playing against itself,
it's gonna be able to learn much more reasoning with itself.
So as soon as the super-intelligence
wants to be the smartest,
it's gonna want more and more resources,
and you're gonna get evolution of super-intelligences.
And let's suppose
there's a lot of benign super-intelligences
who are all out there just to help people.
There are wonderful assistants from Amazon and Google
and Microsoft, and all they want to do is help you.
But let's suppose that one of them
just has a very, very slight tendency
to want to be a little bit better than the other ones.
Just a little bit better.
You're gonna get an evolutionary race
and I don't think that's gonna be good for us.
So I wish I was wrong about this.
I hope that Yann is right,
but I think we need to do everything we can
to prevent this from happening.
But my guess is that we won't.
My guess is that they will take over,
they'll keep us around to keep the power stations running,
but not for long.
'Cause they'll be able to design better analog computers.
They'll be much, much more intelligent
than people ever were.
And we're just a passing stage
the evolution of intelligence.
That's my best guess. And I hope I'm wrong.
But that's sort of a depressing message to close on.
A little bit depressing.
I want to say one more thing,
which is what I call the sentience defense.
So a lot of people think
that there's something special about people.
People have a terrible tendency to think that.
Many people think they, or used to think,
they were made in the image of God.
And God put them in the center of the universe.
Some people still think that.
And many people think
that there's something special about us
that a digital computer couldn't have.
A digital intelligence, it won't have subjective experience.
We're different. It'll never really understand.
So I've talked to philosophers who say,
yes, it understands sort of sub one,
understands in sense one of understanding,
but it doesn't have real understanding
'cause that involves consciousness and subjective experience
and it doesn't have that.
So I'm gonna try and convince you
that the chatbots we have already
have subjective experience.
And the reason I believe that is
'cause I think people are wrong
in their analysis of what subjective experience is.
Okay, so this is a view that I call atheaterism,
which is like atheism.
Dan Dennett is happy with this name
and this is essentially Dan Dennett's view.
He's a well-known philosopher of cognitive science.
It's also quite close to the view of the late Wittgenstein.
Actually, he's dead a long time ago, so he's not that late.
The idea is that most people think that
there's an inner theater,
and so stuff comes from the world,
and somehow gets into this inner theater.
And all we experience directly is this inner theater.
This is a Cartesian kind of view.
And you can't experience my inner theater,
and I can't experience your inner theater.
But that's what we really see.
And that's where we have subjective experience.
That's what subjective experience is
experiencing stuff in this inner theater.
And Dennett and his followers like me,
believe this view is utterly wrong.
It's as wrong as
a religious fundamentalist view of the material world,
which if you're not a religious fundamentalist,
you can agree is just wrong.
And it relies on people not having a very wrong view
of what a mental state is.
So I would like to be able to tell you about
what's going on in my brain when I'm looking at something,
particularly when I'm looking at something and it's weird.
I'd like to tell you I'm seeing this weird thing
that isn't really there, but I'm seeing this weird thing.