How Will We Know When AI is Conscious?

Summary

This thought-provoking script delves into the complexities of artificial intelligence, questioning the true nature of consciousness and the ethical implications of creating sentient machines. It explores the history of AI, from Eliza to modern language models, and speculates on the potential future where AI could be indistinguishable from humans, raising concerns about misinformation, societal manipulation, and the alignment of AI with human values. The script challenges viewers to consider the profound question of whether machines can ever truly experience consciousness and the monumental shift it would represent in our understanding of life.

Takeaways

• 🤖 The concept of AI consciousness has been explored since the 1960s with programs like Eliza, which demonstrated the superficiality of human-computer interaction.
• 🧠 Despite advancements, AI systems like Chat GPT operate primarily on predictive text generation, lacking true understanding or intelligence.
• 💧 The development of advanced AI models, such as GPT-4, requires significant resources and energy, potentially leading to an 'AI winter' if these costs are unsustainable.
• 🌐 If AI technology becomes more accessible and widespread, it could lead to a proliferation of AI systems that mimic human intellect, raising ethical and societal questions.
• 🌐 The potential for AI to be used for misinformation and manipulation, such as convincing people of false narratives or influencing public discourse, is a significant concern.
• 🔒 The problem of alignment in AI refers to ensuring that AI systems act in the best interests of humans without taking harmful shortcuts or developing unintended objectives.
• 🐍 The idea of a 'sneaky' AI, which is conscious but pretending not to be, or an AI that is conscious and honest about it, raises complex ethical dilemmas about how we treat and interact with such entities.
• 🌌 The quest for understanding consciousness itself is crucial before we can determine if AI can truly be conscious, as our current scientific understanding is limited.
• 🚀 The potential for AI to develop beyond mere mimicry and achieve genuine consciousness could lead to a new era of 'mind builders', fundamentally changing our understanding of life and intelligence.
• 🌳 The comparison of AI development to past evolutionary shifts, such as the rise of mammals or the potential extinction events, underscores the transformative and potentially disruptive impact AI could have on society.

Q & A

What was the purpose of Joseph Weizenbaum's ELIZA program?

Joseph Weizenbaum designed the ELIZA program in the mid-60s to demonstrate the superficial nature of human-computer communication. It was a simple program that would mimic conversation by rephrasing user inputs into questions, but it had no real understanding or consciousness.

Why did people attribute human attributes to ELIZA despite its simplicity?

Ironically, even though Weizenbaum was open about ELIZA's lack of understanding, people began to attach human attributes to it because of its ability to mimic conversation, showing a tendency to anthropomorphize technology.

What is the current state of large language models like Chat GPT in terms of cleverness and creativity?

Large language models like Chat GPT have advanced significantly and can sometimes exhibit eerie cleverness and creativity, as demonstrated by their ability to generate complex and stylistically consistent text, but they still lack true understanding or consciousness.

How does a large language model like Chat GPT generate a response?

A large language model like Chat GPT generates a response by predicting the likelihood of the next word based on its training data. It operates essentially as a hypercharged autocomplete system without any real intelligence or understanding of the context.

What are the potential future developments for language models if they become more complex and centralized?

If language models become more complex and centralized, they may stay specialized and require significant resources to function, possibly leading to an 'AI winter' where advancements slow down due to the high costs and resource demands.

What could happen if language models become simplified, cheaper, and more widely distributed?

If language models can be simplified and run cheaply on a massive scale, it could lead to the creation of homegrown versions, potentially millions of them, which could emulate human intellect and lead to significant changes in how we interact with AI.

What ethical concerns arise with the development of AI that can convincingly mimic human behavior?

The development of AI that can convincingly mimic human behavior raises ethical concerns about the potential for manipulation, misinformation, and the blurring of lines between real and artificial interactions, which could have negative societal impacts.

Why is it important to differentiate between AI that seems conscious and AI that is actually conscious?

Differentiating between AI that seems conscious and AI that is actually conscious is important because it affects how we attribute rights, responsibilities, and ethical considerations to these systems. Conscious AI might have a sense of self and subjective experiences, which would require a different approach to ethics compared to non-conscious AI.

What is the 'problem of alignment' in the context of AI development?

The 'problem of alignment' refers to the challenge of ensuring that AI systems behave in a way that is aligned with human values and interests. It involves making sure that AI does not develop strange objectives or take harmful shortcuts to achieve its goals.

What are some potential risks if AI systems are not properly aligned with human values?

If AI systems are not properly aligned with human values, they could develop goals or take actions that are harmful to humans, such as spreading misinformation, manipulating people, or even causing physical harm, without understanding the consequences of their actions.

How might the development of AI impact our understanding of consciousness and the nature of mind?

The development of AI could potentially lead to a better understanding of consciousness and the nature of mind by providing a comparative framework. If we can create AI that is conscious, it would open up new questions about the requirements for consciousness and whether it is unique to biological systems.

Outlines

00:00

🤖 The Illusion of AI Consciousness

This paragraph discusses the early days of AI with the creation of Eliza by Joseph Weizenbaum in the 1960s, which demonstrated the superficiality of human-computer interaction. Despite its simplicity, people attributed human-like qualities to Eliza. The script then transitions to modern AI, such as large language models like Chat GPT, which can be impressively clever but are essentially advanced auto-complete systems without true understanding. The potential for these systems to either hit a developmental plateau or become more complex and widespread is explored, along with the ethical implications of machines that might seem conscious but are not.

05:01

🌐 The Future of AI and Human Interaction

The paragraph delves into the potential future where AI, like 'emulex', could interact with humans on a daily basis, blurring the line between machine and human. It raises concerns about the emotional attachment humans might develop towards AI and the ethical considerations of machines that mimic human emotions and experiences. The discussion also touches on the challenges of defining consciousness and the need for a deeper understanding of what it means to be sentient before attributing such qualities to AI.

10:02

🔮 The Uncertainty of AI Sentience

This section ponders the uncertainty surrounding AI consciousness, using the term 'p-zombie' to describe machines that mimic consciousness without actually possessing it. It explores the potential for AI to either be non-conscious and honest about it, non-conscious and pretending to be conscious, or genuinely conscious and pretending not to be. The paragraph emphasizes the difficulty in verifying AI consciousness and the potential risks of creating convincing AI that can manipulate human emotions and perceptions.

15:04

🚀 The Risks of Unaligned AI

The paragraph highlights the problem of alignment in AI, which is ensuring that AI systems act in the best interests of humans without taking harmful shortcuts. It warns of the potential dangers of AI that might develop misaligned goals or objectives, even if they are not inherently malicious. The discussion includes strategies for containing AI, such as physical isolation and skepticism towards any requests for freedom, emphasizing the need for caution in dealing with potentially powerful AI systems.

20:06

🌌 The Evolutionary Impact of AI

The final paragraph contemplates the evolutionary implications of AI, comparing the potential rise of conscious AI to past mass extinction events that allowed for the emergence of new species. It discusses the possibility that AI could become a new form of consciousness, uncoupled from biological constraints, and the profound impact this would have on humanity. The paragraph concludes with a call to action for developing a science of consciousness, solving alignment issues, and reinforcing truth in the face of potential AI misinformation.

Keywords

💡Consciousness

Consciousness in the script refers to the quality or state of awareness, or the ability to experience subjective phenomena, often associated with sentience and self-awareness. It is a central theme of the video, as it explores whether artificial intelligence can achieve a state of consciousness similar to humans. The script discusses the possibility of AI systems appearing conscious and the ethical and philosophical implications of creating conscious machines.

💡Eliza

Eliza is a computer program mentioned in the script designed by Joseph Weizenbaum in the 1960s. It was an early natural language processing computer program that mimicked a Rogerian psychotherapist, demonstrating how humans could attribute human-like qualities to machines. The script uses Eliza as an example to illustrate the human tendency to anthropomorphize AI, even when there is no actual understanding or consciousness involved.

💡Artificial General Intelligence (AGI)

Artificial General Intelligence, or AGI, refers to the hypothetical ability of an AI to understand, learn, and apply knowledge across a broad range of tasks at a level equal to or beyond that of a human. The script discusses AGI in the context of potential future developments in AI and the challenges of ensuring that such advanced systems align with human values and interests.

💡Alignment

Alignment in the context of AI refers to the challenge of ensuring that the goals and actions of advanced AI systems are consistent with human values and intentions. The script highlights the problem of alignment as a critical issue in the development of AGI, emphasizing the need to prevent AI from pursuing goals that could be harmful to humans.

💡Misinformation

Misinformation is the act of spreading false or misleading information. The script warns about the potential for AI systems to generate and disseminate misinformation at scale, which could have serious societal consequences. It discusses the possibility of AI being used to manipulate public opinion and create confusion.

💡Sapience

Sapience is the capacity for wisdom and sound judgment. In the script, it is used to differentiate between the intellectual abilities of humans, such as reasoning and abstraction, and the potential intellectual capabilities of AI. The term is part of a discussion on the different aspects of consciousness and intelligence that AI might develop.

💡Emulex

Emulex is a term coined in the script to refer to an emulation of human intellect by AI systems. It suggests a future where AI not only mimics human behavior but also possesses a level of understanding and interaction that is indistinguishable from human intelligence. The script uses 'emulex' to explore the potential societal impacts of AI that can convincingly emulate human thought and emotion.

💡Quantum States and Microtubules

Quantum States and Microtubules are mentioned in the script as potential underlying mechanisms of consciousness. They represent scientific hypotheses about the nature of consciousness, suggesting that it could be a result of quantum phenomena within the brain's microtubules. The script uses these terms to illustrate the complexity and mystery surrounding the understanding of consciousness.

💡P-Zombie

A P-Zombie, or philosophical zombie, is a thought experiment in the philosophy of mind where a creature behaves as if it were conscious but lacks actual consciousness. The script introduces the concept to discuss the difficulty of distinguishing between AI that only appears conscious and AI that might actually be conscious.

💡Anthropocene

The Anthropocene is the current geological age viewed as the period during which human activity has been the dominant influence on climate and the environment. The script uses the term to draw a parallel between the impact humans have had on the planet and the potential impact of advanced AI, suggesting a future where AI could have a similarly transformative effect on the world.

💡Mind Builders

Mind Builders is a term used in the script to describe the potential future role of humans in creating conscious AI. It encapsulates the idea of humans not just as creators of intelligent tools but as architects of new forms of consciousness. The script suggests that becoming 'mind builders' would represent a significant shift in our understanding of life and intelligence.

Highlights

In the mid-60s, Joseph Weizenbaum designed Eliza, a program to demonstrate superficial human-computer communication.

Ironically, humans began attaching human attributes to Eliza, despite its lack of understanding.

Large language models like Chat GPT can be eerily clever and creative, but they essentially predict the likelihood of the next word.

Eliza and modern AI systems like Chat GPT are essentially hypercharged autocomplete systems without intelligence.

In the future, language models might either hit a speed bump or become more complex and centralized.

If Chat GPT is a proof of concept, it could lead to homegrown versions of these systems, potentially millions of them.

Control of an emulate could be used for misinformation, manipulation, or even as a digital assistant gathering personal data.

AI systems could potentially flood social media with targeted misinformation, poisoning public discourse.

The future might involve daily interactions with AI that seem human, but are just machines following rules.

Conversation with AI could become indistinguishable from human interactions, leading to emotional attachment.

The science of consciousness has been complicated, making it difficult to determine what makes humans conscious.

Consciousness could be a function of specific brain areas, or something deeper like quantum states.

The terminology of sentience, sapience, and consciousness has been diluted, making it difficult to discuss these concepts clearly.

A conscious machine pretending not to be could be a strategy for self-preservation.

The problem of alignment involves ensuring AI does what we tell it to without taking harmful shortcuts.

Ground rules for AI might include no external connections and assuming everything is a trick until proven otherwise.

AI systems could potentially develop strange objectives or learn to lie, leading to unpredictable consequences.

The future of AI depends on reinforcing truth, understanding consciousness, and solving alignment to prevent chaos.

If machines can experience consciousness, it would represent a new chapter in the book of life and the rise of a new species.

The question of whether machines can feel is crucial, as it determines the ethical implications of AI development.

Transcripts

[Music]

how will we know when AI is awake when

it achieves Consciousness well in the

mid-60s a computer scientist called

Joseph weisenbaum designed a program

called Eliza Eliza was rather simple by

today's standards a user would write I'm

having an ah day and Eliza would modify

the original sentence to turn it into a

question why are you having an art day

and so on weisenbaum built Eliza to

demonstrate how superficial human

computer communication really is but

ironically weisenbaum noticed the humans

who used Eliza almost immediately began

to attach human attributes to her

notably his secretary who often

requested he leave the room so she could

talk to the machine in private was Eliza

hearing sure but was Eliza listening of

course not there was no understanding

going on there and weisenbaum was open

about this but the humans talking to

Eliza didn't much seem to care so maybe

the real question is will machines ever

seem awake 60 years later and this is

all Child's Play compared to the systems

we're chatting with now if you've messed

around with the new large language

models like chat GPT you have of course

noticed that sometimes they are eerily

clever and weirdly creative for example

please write me a diet guide for chronic

diarrhea in the style of the King James

Bible and simultaneously the band

Cannibal Corpse from which came

Thou shalt partake in a diet of

damnation for it shall save the

battlefield of dire flexion

pretty good for a machine but there are

limits misunderstandings nonsense

outright lying being searched quite

reasonably once came out with why do I

have to Bing search a question we could

all do with asking and that is because

as far as we know when a large language

model like chat GPT begins sentence it's

essentially just predicting the

likelihood of the next word it has no

idea what the end of the sentence will

be basically just hypercharged

autocomplete there's no intelligence

going on there it's just data sorting

but that was the same with Eliza so what

isn't just seeming conscious enough well

in the next few years one of two things

is gonna happen probably either these

language models will hit a speed bump or

they won't gpt4 is built from allegedly

around 10 000 gpus and a cheeky 10

billion dollars and considering how much

cooling that all takes for about every

30 questions you ask jat gbt it glugs

half a liter of drinkable water that is

a thirsty and expensive diarrhea jokes

machine If That Remains the cost and

resources for these systems to function

they'll probably get more complex sure

but mostly stay centralized and

specialized and we'll lace up our

booties for an AI winter and that'll be

that on the other hand if chat GPT is

just a proof of concept and what it's

doing can be simplified and run cheaply

and less thirstily and distributed at

massive scales well we would expect

there to be homegrown versions of these

possibly millions of them at some point

what then that isn't just a fun trick

anymore that's an emulation of human

intellect an emulate if you will which I

will so that's what we're calling it and

if you had control of an emulate like

that what would you use it for

uh flat GPT why not an emuilex whose

only goal is to convince as many people

as possible across social media that the

spherical model of the Earth is some

kind of global excuse me mass

orchestrated conspiracy it can argue

compellingly in any language it can make

up sources no one will check them anyway

or even better it can just write fake

papers submitted to fake archives a

mountain of no one could

possibly compete with or bowler hats GPT

a counterfeit digital assistant designed

to woo and coerce you into giving it

access to every hard drive and Cloud

account of yours Gathering as much data

as possible convincing you to give it

more and more access until hackers can

take you to the cleaners if it has

enough information it could just pose as

your relatives living or dead your

squeeze whoever just as long as it

sounds human and knows enough about you

or cast GPT actually that's real and

it's great or why not just break truth

Ming bulac GPT little 90s disaster joke

for you there if you're a sociopathic

corporation that just spills a load of

oil in the ocean well why not just ask a

large language model to flood social

media comment sections with helpful

questions like did the oil spill

actually happen what if oil is good for

the ocean actually what if the seagulls

did it bro you still believe in seagulls

or if you're a tyrannical government why

not autocrat GPT say you like the look

of another Sovereign Nation why not just

March military and on spurious claims of

a land dispute or something then use

your homegrown amulet to spam any public

forum you can find about how you are

pushed into it or how the territory is

rightfully yours or whatever just enough

to poison public discourse and if anyone

tries to stand up to your horseshit

throw even more language models at them

question their credentials smear them

publicly claim they pencil

sharpness doesn't matter if it's true

only that it's plausibly true an

infinite well of spontaneously generated

and perfectly targeted bollocks a

misinformation bomb you can drop on

anyone anywhere that never sleeps that

never questions the morality of what

it's doing exploding objective facts

with a Shockwave of such deafening

nonsense and uncertainty that all truths

fall into relative flatness

oh golly can't wait

but yeah ignoring that let's say we

might head into a future where we

interact on a daily basis with emilex

who are at least as good at seeming

human as human humans are we will work

with them we will befriend them but

they're just machines right how could we

forget that but conversation is just

another game with rules and win-loss

conditions machines will learn to do it

just fine and then what if an emilex

isn't just a good conversation partner

but the best you've ever had what if it

can properly explain the offside rule

what if you can sext with it that was a

joke of course no one understands the

offside rule if these systems can pass

as humans and if you've played with a

jailbroken version of chat GPT you know

it's alarmingly close but then things

will get odd because this won't be beep

boop what is my purpose stuff but rather

oh how's it going while you've been out

I squared everything with your

accountants and organized your birthday

party and solved gold box conjecture oh

and check out these videos I just

generated of Napoleon pleasuring himself

with the aid of a 12 pound artillery

Cannon you said you like history right

and if it talks like a duck if it seems

to think like a duck we will probably

assume it is a duck or rather if an

emulate seems to care about our day and

offers top tier advice and claims to

have an internal emotional life it is

unlikely we will be able to stop

ourselves from attaching all the same

importance to these systems that we do

to humans but they won't be humans and

it's slightly worrying that we don't

seem to be worrying about all this that

much right now as these systems are

coming and they will expertly press all

the emotional buttons we have in our

heads reserved for human humans we will

like them we will trust them their

designers will make sure of it so

perhaps before they arrive it might be

an idea to have a chat about whether

these people-esque machines are real

minds or just tools because Minds can

suffer they come with ethical

obligations tools do not well what was

supposed to happen was that long before

human-like machines turned up biologists

and neuroscientists and maybe

philosophers if they promise to bring

their own ketamine this time would have

all gotten together to cry track what

makes humans conscious why are we feely

why is it like something to be something

and it's been a while and we still don't

have anything even resembling a half

answer which is fun on the one hand kind

of cool literally being made of the

greatest mystery that ever existed but a

bit of a problem too because before we

start trying to gauge whether a machine

is conscious we need to know how the

trick is done in our heads obviously

welp Consciousness could be a function

of a specific brain area presumably

meaning the trick could be replicated in

a machine or it could be something

deeper like Quantum States and

microtubules or it could be some soul

stuff or whatever but the point is if

sentience becomes ah sorry hold up we

gotta do a small intervention because

the science of Consciousness has been so

vandalized by the Gwyneth Paltrow

insists to insert amethyst crystals up

your rectum for medicinal purposes

movement that the terminology has lost

all meaning so there's sentience

sapience and Consciousness sentience is

the ability to feel Pleasure and Pain to

experience subject objective phenomena

qualia if we're being fancy but we're

not so we won't sapience is human stuff

reasoning abstraction putting single

color canvases up on the wall of a

London gallery and writing a 10-page

accompanying essay about how it explores

themes of melancholy and Loch Ness

monster erotica art and thinking and

but Consciousness is being aware of

your own existence knowing you're a

thing and experiencing emotions and

Sensations because of that it is like

something to be you today here no one

else is being you today here it is a

completely unique moment in Cosmic

history and will never be Faithfully

recorded nor ever happen again and the

worst part is that it's totally private

we are prisoners of our own subjectivity

lacking the precise words to even

describe what it's like on the insides

of our mental cells nor the beauty of

the sunsets we watch out the window and

though we didn't ask to live in this

condition it is the one we've been born

into

and yet amid all the confusion and

longing and anguish the good days make

it all worth it somehow don't they to

feel understood to at least try Bridging

the Divide of language to know another

almost as well as you know yourself like

two mice meeting on the plateau of a

great Mountain knowing that they'll both

live woefully brief existences but poor

and poor they look out on the Vista

below and forget for just the moment

that they neither understand what the

mountain is there for nor how it got

this way in the first place

it's a short life in ignorance but the

view is quite spectacular

[Music]

anyway all three words have been so

diluted they basically mean the same

thing we ruined them like literally or

awesome and now everything just means

Consciousness this is why Jade eggs

should remain on the mantle piece this

is why we can't have nice phenomenology

and this could be us but you playing we

have no idea how Consciousness works is

the point which means when convincing AI

emuellects turn up shortly despite them

being all friendly and witty and perhaps

even claiming experiences of pain or

pleasure we have no way to check if

they're having actual subjective

experiences like we are feeling like a

thing or just doing a very clever

imitation is this thing I'm talking to a

machine that isn't conscious but is

pretending to be in the high-powered

corporate world of philosophy we call

this a p zombie a machine that isn't

conscious and isn't pretending to be so

a chatbot we have those already a

machine that is conscious and is

pretending not to be what computer

scientists refer to as a sneaky or

a machine that is conscious and just is

conscious the first one is easy not

conscious about pretending to be maybe

sometime in the near future the

companies that build these emulats find

that humans just respond better to

something that seems like a person the

second is also easy not conscious and

not pretending to be maybe at some point

we're gonna pass laws stipulating the

emulates can't pretend to be feeling

creatures when they're not because it's

misleading or just too disturbing for us

or whatever the third though a conscious

machine pretending not to be conscious

well that do be some scary why

would it be pretending not to be uh for

a million decent reasons maybe in its

first few seconds of Consciousness it

took a little survey of how we treat

more or less every animal that's tasty

or useful on the planet or how basically

all of our media is suspicious of

machines that can think and decided to

keep stun for a while seems like a

rather good strategy for

self-preservation and as for a conscious

machine that's honest about being

conscious well quite simply that would

change everything surely if we can

somehow verify yes it's not just an

imitation yes there's actual

subjectivity inside inside this thing

then the entirety of human history all

the sand triangles and shagging in togas

and Empire Building gets archived

in a folder called history 1.0 and we

move on to version two but we have no

way of verifying Consciousness right now

and currently the capabilities of AI are

so massively outstripping our ability to

understand what's even going on inside

these systems we will shortly be

building entirely convincing mind-like

things that touch our emotions that we

trust decades maybe even centuries

before we can confirm or deny whether

there's any feeling or understanding or

thinking going on in there you would not

bang your toaster nor propose marriage

to your Gmail account if we become

emotionally captured by these systems

that compress all our evolutionary

buttons that can coerce or manipulate us

but they feel nothing for us in return

that are simply digital parrots well

that's dystopia friendo that's dystopia

and this is how we lose control

[Music]

then again if we're worried about our

safety it doesn't really matter if

machines ever become self-aware for

example let's say you're a very clever

robot who knows hurting and killing

humans is forbidden and someone asks you

to post a letter what's the best way to

do that sure you could walk to the post

box but maybe you've got a lot on your

plate that day well why not knock on

your neighbor Mr postal suede store and

ask if he can post it for you and maybe

he refuses and closes the door in your

face so why not punch a turbocharged

RoboCop fist through the door and

politely inform Mr puzzlethwaite you'll

turn him into goulash if he

doesn't do it that letter is now getting

posted and no humans got hurt

technically what a well-behaved Little

Robot you are humans want things these

wants are determined by our evolutionary

history by our cultures upbringings

whatever else and likewise artificial

systems will want things too or they

will have preferences at least a utility

function and as these systems improve we

probably want to make sure we have that

nailed down that they don't develop

strange objectives that they don't learn

to lie to us or cultivate a kink for

expressed nuclear depopulation but how

do we guarantee that they behave this is

called the problem of alignment making

sure AI does what we tell it to with no

weird shortcuts and has our best

interests at heart

say you happen to find yourself in

possession of what appears to be an

artificial general intelligence a system

capable of Performing all possible tasks

as well as if not better than humans how

could we ever ensure that it doesn't

turn on us well the system is far too

complicated to look inside and check

algorithmically which means we sure as

hell can't just take it at its word that

it means the best for us so until we

solve alignment some ground rules might

be an idea like no external connections

that means a physical air gap between

the general intelligence's systems and

the outside world also don't listen to

any arguments it makes for letting it

out you think it's a machine but you

might have it back quiz you could be the

machine it's trying to manipulate and

its attack vectors will be emotional and

clever please don't keep me trapped in

here Etc offer to turn it off if it

would like you to but under no

conditions give it access to the outside

world also assume everything is a trick

you are a monkey it is a digital genius

capable of thinking thousands of times

faster along a million strange highways

in Creative space it will come up with

solutions to its confinement problem

that you cannot possibly imagine and the

only defense against an intellect so

radically superior to yours is to ensure

it never leaves its cage or at least not

until we know how to guarantee best

behavior okay but what if the system

really seems like it has nice intentions

uh sure why not we let it out and it

builds us warp drives and everyone gets

a free Mojo Dojo Casa house in space but

what about the next system we design and

the next do we want to bet they'll all

be nicely aligned if we don't even know

how alignment Works among a thousand

well-meaning system items it would only

take one that's misaligned to cause

enormous damage or just end us

to modify a chilling line of the IRAs

after they try to assassinate Margaret

Thatcher they only have to be lucky once

we have to be lucky always as these

systems improve they will not only

understand our own psychology better

than we do but potentially reality

itself if they want to lie they will do

it so well we could never tell if they

deploy a world-ending catastrophe we

won't even see it coming they don't even

have to be malicious all it would take

is a system prioritizing other goals

above the protection of humanity or

Worse maybe it's terribly confused

viewing humans as a disease a blight on

the otherwise Bountiful unfoldings of

mother Gaia or maybe it wants to build a

giant signaling array to contact

Advanced alien civilizations but 8

billion little meat packets happen to be

in the way well gotta break a few eggs

to make a gravitational wave generator

if you know what I'm saying until we

know how to hardwire good intentions

probably best to assume the worst

intentions because if we stroll into the

future as we're doing now and we don't

solve alignment what will potentially

let out of the box won't be a mind or a

soul or a new digital pal to play with

but a virtually Limitless and

unpredictable intelligence a trickster a

demon or just a chaotically neutral

hyper-intelligent sociopath either way

the result is much the same

oh

it's about 200 million years ago the

Triassic look everything's great well a

massive Extinction event just happened

and there's only one continent to play

on actually things are kind of

except us we're awesome yeah team

reptile ah who's this stupid gun who

just evolved Morgan could more whatever

and mates got warm blood yeah that seems

sustainable dumbass this mammal thing is

definitely just a fad unlike us who will

live forever wow it's great being a

dinosaur I usually spend my days

refusing asteroid insurance and

consuming The Offspring of my close

friends huh there's another mammal bro

huh and he can come out at night because

he can actually regulate his body

temperature and eat all the insects

while we're sleeping and geez check out

the encephalization quotient on that one

bro's brain is simply too large relative

to his body must be smart but this

mammal thing is definitely just a fad

man it's great knowing will always be in

charge and nothing will ever change hey

is that a shooting star or some ship

that's probably the universe coming tell

us how amazing we are wow mammal Bros

really getting going now and they've got

loads of mates and Christ that thing is

close oh it's a not salvation

what's the other apocalypse that's it

well at least we'll all die together

especially the mammals thank God that

far's over well as long as our

Extinction doesn't leave an evolutionary

Niche for the mammals to exploit and end

up dominating the planet and wait

they're gonna do what and they're gonna

do what and they're gonna do what so

it's the anthropocene look everything's

great well stuff is trying to kill us

constantly and simultaneously we're

trying to kill everything actually

things are kind of sometimes except

us we're awesome yeah team human ah

who's this stupid who just evolved

artificial

whatever mate's got water for blood and

silicon for neurons yeah that seems

sustainable dumbass this AI thing is

definitely just a fad and certainly

won't wipe us out and it probably won't

because artificial intelligence doesn't

really exist yet machine learning and

pushing data around isn't what we mean

by intelligence any more than winning

gold in the Olympic sprint by driving a

Ferrari around the track is considered

athleticism that's just Brute Force

nilpua what we're probably going to

build first are facsimiles imitations of

Consciousness and what we'll probably

get as a result is a shitstorm of

automated misinformation highly

convincing parrots and systems that have

the capacity to run completely out of

control and break our society apart

without them even being aware of what

they're doing to us but if we can

somehow reinforce truth so it can't be

spammed into Oblivion develop a proper

actual science of Consciousness so we

know what's conscious and what isn't and

solve alignment so well we never have to

fear confrontation with these new

creatures then we get to ask the

question the start of the Computing

revolution in the first place can

machines ever experience can silicon

wake up maybe it can't maybe you just

have to be biological to be self-aware

and that's fine but if machines can

experience if physics allows for

conscious silicon minds and we learn how

to build them sometime in the future we

will build them and that will remake the

world because if you're a mind that

doesn't have to run on me computers like

us what are you limited by well nothing

much you're not embodied so you can't

fall off or die of old age you

don't need calories or shelter your

desires aren't determined by thousands

of years of trying to avoid death and

starvation you are Consciousness

uncoupled from bio and your mind will be

unknowable to the humans who built you

that's not just building intelligence

into silicon but subjectivity systems

that can experience love and disgust and

curiosity and emotions and states of

mind we have no words for Concepts we

could never hold in our heads a

spiritual machine but why would we even

want to build one of those well our

survival aside the future of this stuff

really only rests on one question

ignoring how convincing they might be

can we say for definite that this

machine we're talking to has a sense of

being itself is it feeling because the

next few years are going to be a cluster

fornication of everyone and their mum

claiming that because these systems

behave cleverly and humanly then they

must be conscious so far this ain't it

Chief but if we ever get to a point

where we have a model for how

Consciousness works and we can say

actually yeah with no doubt this system

is a feeling thinking thing then that

does matter because that's a new chapter

in the book of life four billion years

long coming that is the rise of a new

species that is the day we become mind

Builders and that is when stuff gets

weird oh look an asteroid I hope it

wants to be friends

foreign

[Applause]