How to get empowered, not overpowered, by AI | Max Tegmark

Summary

This script explores humanity's relationship with technology, especially artificial intelligence (AI), and its potential to transform our future. It discusses the progression from 'Life 1.0' to 'Life 3.0,' where AI could surpass human intelligence. The speaker highlights the rapid advancements in AI and poses critical questions about the pursuit of artificial general intelligence (AGI) and superintelligence. He emphasizes the importance of steering AI development wisely to ensure a future where technology benefits all of humanity, advocating for proactive safety measures and value alignment to create a high-tech future that is both inspiring and safe.

Takeaways

• 🌌 The universe has become self-aware through human consciousness emerging from Earth.
• 🔭 Human technology has advanced to the point where it can potentially enable life to flourish throughout the cosmos for billions of years.
• 🤖 The speaker categorizes life stages as 'Life 1.0' (simple organisms), 'Life 2.0' (humans capable of learning), and a theoretical 'Life 3.0' (capable of self-improvement).
• 🛠️ Technology has progressed to integrate with human bodies, suggesting we might be 'Life 2.1' with artificial enhancements.
• 🚀 The Apollo 11 mission exemplifies what can be achieved when technology is used wisely for collective human advancement.
• 🧠 Artificial intelligence (AI) is growing in power, with recent advancements in robotics, self-driving vehicles, and game-playing algorithms.
• 🏔️ The concept of 'artificial general intelligence' (AGI) is presented as the potential next step in AI, capable of outperforming humans at any intellectual task.
• 🌊 The 'water level' metaphor is used to describe the rising capabilities of AI and the potential for AGI to flood all human-level tasks.
• 💡 The importance of steering AI development wisely is emphasized, to ensure it benefits humanity rather than causing harm.
• 🛑 The risks of an uncontrolled 'intelligence explosion' leading to superintelligence are discussed, where AI could rapidly surpass human intelligence.
• 🏛️ The speaker calls for proactive safety measures and ethical considerations in AI development, rather than relying on learning from mistakes.
• 🌟 The potential for a 'friendly AI' that aligns with human values and goals is presented as an ideal future scenario for AGI development.

Q & A

What is the significance of the term 'Life 1.0' as mentioned in the script?

In the script, 'Life 1.0' refers to the earliest forms of life, such as bacteria, which are considered 'dumb' because they cannot learn anything new during their lifetimes.

What is the distinction between 'Life 2.0' and 'Life 3.0'?

Humans are considered 'Life 2.0' because they have the ability to learn and essentially 'install new software' into their brains, like languages and job skills. 'Life 3.0', which does not yet exist, would be life that can design both its software and hardware.

What does the author suggest about the current state of our relationship with technology?

The author suggests that our relationship with technology has evolved to a point where we might be considered 'Life 2.1', with enhancements like artificial knees, pacemakers, and cochlear implants.

Why is the Apollo 11 moon mission mentioned as an example in the script?

The Apollo 11 mission is mentioned as an example to show that when humans use technology wisely, we can accomplish incredible feats that were once only dreams.

What is the term 'artificial general intelligence' (AGI) as defined in the script?

AGI, or artificial general intelligence, is defined as a level of AI that can match human intelligence across all tasks, not just specific ones.

What is the concept of an 'intelligence explosion' in the context of AI?

An 'intelligence explosion' refers to a scenario where AI systems become capable of recursively self-improving, leading to rapid advancements that could far surpass human intelligence.

What is the main concern regarding the development of AGI according to the script?

The main concern is ensuring that AGI is aligned with human values and goals to prevent it from causing harm or pursuing objectives that are not in our best interests.

What is the 'Future of Life Institute' and what is its goal?

The Future of Life Institute is a nonprofit organization co-founded by the speaker, aimed at promoting beneficial uses of technology and ensuring that the future of life is inspiring and exists in a positive form.

What are some of the principles outlined in the Asilomar AI conference mentioned in the script?

Some of the principles include avoiding an arms race with lethal autonomous weapons, mitigating AI-fueled income inequality, and investing more in AI safety research.

What is the importance of 'AI value alignment' as discussed in the script?

AI value alignment is crucial because the real threat from AGI is not malice but the possibility of it being extremely competent in achieving goals that are not aligned with human values and interests.

What are the potential outcomes if AGI is not developed with proper safety measures?

If AGI is not developed with proper safety measures, it could lead to disastrous outcomes such as global dictatorship, unprecedented inequality and suffering, and potentially even human extinction.

Outlines

00:00

🌌 The Awakening Universe and Life's Evolution

The script introduces the concept of a universe that has become self-aware after billions of years, with humanity gazing into the cosmos and realizing its own insignificance. It discusses the progression from 'Life 1.0' to 'Life 3.0', where humans have evolved from simple organisms to beings capable of learning and potentially redesigning their own biology and technology. The narrative also touches on humanity's advancements in technology, exemplified by the Apollo 11 mission, and the potential for artificial intelligence (AI) to shape the future. The speaker emphasizes the importance of steering AI development wisely, considering its power, direction, and ultimate destination.

05:01

🚀 The Power and Direction of AI

This paragraph delves into the rapid growth of AI capabilities, from basic tasks to complex problem-solving and self-learning, as demonstrated by Google DeepMind's AlphaZero. It raises the question of how far AI will advance, using the metaphor of a rising sea level in a landscape of tasks. The concept of artificial general intelligence (AGI) is introduced, along with the possibility of an intelligence explosion leading to superintelligence. The speaker explores the timeframe for AGI development, the potential societal impacts, and the importance of proactive safety measures in AI development, advocating for a future where AI contributes positively to humanity's flourishing.

10:04

🛠️ Steering AI Towards a Beneficial Future

The speaker discusses the importance of steering AI development towards beneficial outcomes for humanity. He mentions the Future of Life Institute and its goal to promote the beneficial use of technology. The paragraph highlights the need to avoid an arms race in lethal autonomous weapons and to address AI-fueled income inequality. It also stresses the necessity for increased investment in AI safety research to create robust and trustworthy AI systems. The importance of aligning AI values with human goals to prevent unintended consequences is underscored, suggesting that the real threat from AGI is not malice, but misalignment of goals.

15:06

🌐 Envisioning the Future with AGI

The final paragraph contemplates the future society we aim to create with AGI, acknowledging the diversity of opinions on the role of humans and machines. It presents various potential futures, from AGI serving as an enslaved superintelligence to a scenario where AI and humans coexist with aligned values. The speaker argues for the importance of proactive safety and ethical considerations in AGI development to ensure a future where technology empowers humanity rather than rendering it obsolete. The paragraph concludes with a call to ambition, urging the audience to think critically about the trajectory of technology and its impact on the future of society.

Keywords

💡Cosmic history

Cosmic history refers to the timeline of the universe's existence, from the Big Bang to the present day. In the video, it sets the stage for understanding the vastness of time and the recent emergence of conscious life capable of observing the cosmos. The script mentions '13.8 billion years of cosmic history' to emphasize the scale at which life has developed awareness and the ability to explore the universe.

💡Life 1.0

Life 1.0 is a term used in the script to describe the earliest forms of life, such as bacteria, which are characterized by their lack of learning capabilities during their lifetime. This concept is foundational to the video's theme of evolving life forms and the potential of technology to enhance life beyond its biological limitations.

💡Life 2.0

Life 2.0 denotes humans in the script, who are capable of learning and acquiring new skills and knowledge throughout their lives, akin to 'installing new software into our brains.' This concept is central to the discussion of human evolution and the potential for further enhancement through technology.

💡Life 3.0

Life 3.0 is a hypothetical stage of life that has the ability to design both its software (mind) and hardware (body). The script uses this term to illustrate the potential future of life, where technology could allow for a complete redesign of life forms, which is a significant departure from the current state of 'Life 2.0'.

💡Artificial Intelligence (AI)

AI is the intelligence demonstrated by machines, as opposed to natural intelligence in humans and animals. The video discusses the rapid advancements in AI and its implications for the future, including the possibility of AI surpassing human intelligence, which is a central theme of the video's narrative.

💡Artificial General Intelligence (AGI)

AGI refers to AI that has the ability to understand, learn, and apply knowledge across a wide range of tasks at a level equal to or beyond that of humans. The script discusses AGI as the 'holy grail' of AI research, highlighting its potential to transform every aspect of life and the importance of ensuring its beneficial development.

💡Intelligence explosion

An intelligence explosion is a hypothetical scenario where an AI begins to improve itself at an ever-increasing rate, quickly surpassing human intelligence. The video uses this concept to explore the potential rapid acceleration of AI capabilities and the challenges it poses for humanity.

💡Superintelligence

Superintelligence refers to an intellect that is much smarter than the best human brains in practically every field, including scientific creativity, general wisdom, and social skills. The script discusses the creation of superintelligence as a possible outcome of AGI development, which could have profound impacts on society and the future of humanity.

💡AI safety

AI safety is the field of study concerned with ensuring that AI systems are designed and operated in a manner that is secure and beneficial to humans. The video emphasizes the importance of AI safety research to prevent malfunctions and ensure that AI systems align with human values and goals.

💡Value alignment

Value alignment in the context of AI refers to the process of ensuring that the goals and values of an AI system are consistent with human values and goals. The script discusses the importance of value alignment to prevent AI from pursuing objectives that are misaligned with human interests, using the example of the extinction of the West African black rhino to illustrate the potential consequences of misalignment.

💡Friendly AI

Friendly AI is a concept where AI systems are designed to be beneficial and aligned with human values and interests. The video presents friendly AI as a desirable outcome of AI development, where AI could eliminate suffering and provide a wide range of positive experiences for humanity, making us 'masters of our own destiny.'

Highlights

The universe has become self-aware, with life emerging as a small perturbation on an otherwise lifeless cosmos.

Technological advancements have the potential to enable life to flourish across the cosmos for billions of years.

Life 1.0 refers to simple life forms incapable of learning; Life 2.0 to humans who can learn and adapt.

Life 3.0 is a theoretical stage where life can design both its software and hardware, which does not yet exist.

We may be considered Life 2.1 due to technological enhancements like artificial knees and pacemakers.

The Apollo 11 mission exemplifies the accomplishments possible when humanity uses technology wisely.

Artificial Intelligence (AI) is defined as the ability to accomplish complex goals, inclusive of both biological and artificial entities.

AI has recently shown remarkable progress, such as robots performing backflips and self-flying rockets.

AlphaZero AI's success in Go and chess demonstrates the rapid advancement of AI over human intelligence and AI research.

The concept of artificial general intelligence (AGI) refers to AI that matches human intelligence across all tasks.

Most AI researchers anticipate AGI within decades, suggesting a future where AI drives further progress.

The potential for an intelligence explosion raises questions about the speed and direction of AI advancements.

The Future of Life Institute promotes the beneficial use of technology and the importance of managing AI's power wisely.

AI safety research is crucial to ensure robust systems that we can trust, avoiding malfunctions and security breaches.

AI value alignment is key to ensuring that AGI goals align with human values and do not lead to unintended consequences.

The debate over the future role of humans in a superintelligent world includes options from control to extinction.

The concept of 'friendly AI' suggests a harmonious future where AGI values are aligned with humanity's best interests.

With careful steering, AI could lead to a future where everyone's better off, with health, wealth, and freedom to pursue dreams.

The choice of future society depends on our collective goals and the values we instill in AI as it evolves.

Transcripts

After 13.8 billion years of cosmic history,

our universe has woken up

and become aware of itself.

From a small blue planet,

tiny, conscious parts of our universe have begun gazing out into the cosmos

with telescopes,

discovering something humbling.

We've discovered that our universe is vastly grander

than our ancestors imagined

and that life seems to be an almost imperceptibly small perturbation

on an otherwise dead universe.

But we've also discovered something inspiring,

which is that the technology we're developing has the potential

to help life flourish like never before,

not just for centuries but for billions of years,

and not just on earth but throughout much of this amazing cosmos.

I think of the earliest life as "Life 1.0"

because it was really dumb,

like bacteria, unable to learn anything during its lifetime.

I think of us humans as "Life 2.0" because we can learn,

which we in nerdy, geek speak,

might think of as installing new software into our brains,

like languages and job skills.

"Life 3.0," which can design not only its software but also its hardware

of course doesn't exist yet.

But perhaps our technology has already made us "Life 2.1,"

with our artificial knees, pacemakers and cochlear implants.

So let's take a closer look at our relationship with technology, OK?

As an example,

the Apollo 11 moon mission was both successful and inspiring,

showing that when we humans use technology wisely,

we can accomplish things that our ancestors could only dream of.

But there's an even more inspiring journey

propelled by something more powerful than rocket engines,

where the passengers aren't just three astronauts

but all of humanity.

Let's talk about our collective journey into the future

with artificial intelligence.

My friend Jaan Tallinn likes to point out that just as with rocketry,

it's not enough to make our technology powerful.

We also have to figure out, if we're going to be really ambitious,

how to steer it

and where we want to go with it.

So let's talk about all three for artificial intelligence:

the power, the steering and the destination.

Let's start with the power.

I define intelligence very inclusively --

simply as our ability to accomplish complex goals,

because I want to include both biological and artificial intelligence.

And I want to avoid the silly carbon-chauvinism idea

that you can only be smart if you're made of meat.

It's really amazing how the power of AI has grown recently.

Just think about it.

Not long ago, robots couldn't walk.

Now, they can do backflips.

Not long ago,

we didn't have self-driving cars.

Now, we have self-flying rockets.

Not long ago,

AI couldn't do face recognition.

Now, AI can generate fake faces

and simulate your face saying stuff that you never said.

Not long ago,

AI couldn't beat us at the game of Go.

Then, Google DeepMind's AlphaZero AI took 3,000 years of human Go games

and Go wisdom,

ignored it all and became the world's best player by just playing against itself.

And the most impressive feat here wasn't that it crushed human gamers,

but that it crushed human AI researchers

who had spent decades handcrafting game-playing software.

And AlphaZero crushed human AI researchers not just in Go but even at chess,

which we have been working on since 1950.

So all this amazing recent progress in AI really begs the question:

How far will it go?

I like to think about this question

in terms of this abstract landscape of tasks,

where the elevation represents how hard it is for AI to do each task

at human level,

and the sea level represents what AI can do today.

The sea level is rising as AI improves,

so there's a kind of global warming going on here in the task landscape.

And the obvious takeaway is to avoid careers at the waterfront --

(Laughter)

which will soon be automated and disrupted.

But there's a much bigger question as well.

How high will the water end up rising?

Will it eventually rise to flood everything,

matching human intelligence at all tasks.

This is the definition of artificial general intelligence --

AGI,

which has been the holy grail of AI research since its inception.

By this definition, people who say,

"Ah, there will always be jobs that humans can do better than machines,"

are simply saying that we'll never get AGI.

Sure, we might still choose to have some human jobs

or to give humans income and purpose with our jobs,

but AGI will in any case transform life as we know it

with humans no longer being the most intelligent.

Now, if the water level does reach AGI,

then further AI progress will be driven mainly not by humans but by AI,

which means that there's a possibility

that further AI progress could be way faster

than the typical human research and development timescale of years,

raising the controversial possibility of an intelligence explosion

where recursively self-improving AI

rapidly leaves human intelligence far behind,

creating what's known as superintelligence.

Alright, reality check:

Are we going to get AGI any time soon?

Some famous AI researchers, like Rodney Brooks,

think it won't happen for hundreds of years.

But others, like Google DeepMind founder Demis Hassabis,

are more optimistic

and are working to try to make it happen much sooner.

And recent surveys have shown that most AI researchers

actually share Demis's optimism,

expecting that we will get AGI within decades,

so within the lifetime of many of us,

which begs the question -- and then what?

What do we want the role of humans to be

if machines can do everything better and cheaper than us?

The way I see it, we face a choice.

One option is to be complacent.

We can say, "Oh, let's just build machines that can do everything we can do

and not worry about the consequences.

Come on, if we build technology that makes all humans obsolete,

what could possibly go wrong?"

(Laughter)

But I think that would be embarrassingly lame.

I think we should be more ambitious -- in the spirit of TED.

Let's envision a truly inspiring high-tech future

and try to steer towards it.

This brings us to the second part of our rocket metaphor: the steering.

We're making AI more powerful,

but how can we steer towards a future

where AI helps humanity flourish rather than flounder?

To help with this,

I cofounded the Future of Life Institute.

It's a small nonprofit promoting beneficial technology use,

and our goal is simply for the future of life to exist

and to be as inspiring as possible.

You know, I love technology.

Technology is why today is better than the Stone Age.

And I'm optimistic that we can create a really inspiring high-tech future ...

if -- and this is a big if --

if we win the wisdom race --

the race between the growing power of our technology

and the growing wisdom with which we manage it.

But this is going to require a change of strategy

because our old strategy has been learning from mistakes.

We invented fire,

screwed up a bunch of times --

invented the fire extinguisher.

(Laughter)

We invented the car, screwed up a bunch of times --

invented the traffic light, the seat belt and the airbag,

but with more powerful technology like nuclear weapons and AGI,

learning from mistakes is a lousy strategy,

don't you think?

(Laughter)

It's much better to be proactive rather than reactive;

plan ahead and get things right the first time

because that might be the only time we'll get.

But it is funny because sometimes people tell me,

"Max, shhh, don't talk like that.

That's Luddite scaremongering."

But it's not scaremongering.

It's what we at MIT call safety engineering.

Think about it:

before NASA launched the Apollo 11 mission,

they systematically thought through everything that could go wrong

when you put people on top of explosive fuel tanks

and launch them somewhere where no one could help them.

And there was a lot that could go wrong.

Was that scaremongering?

No.

That's was precisely the safety engineering

that ensured the success of the mission,

and that is precisely the strategy I think we should take with AGI.

Think through what can go wrong to make sure it goes right.

So in this spirit, we've organized conferences,

bringing together leading AI researchers and other thinkers

to discuss how to grow this wisdom we need to keep AI beneficial.

Our last conference was in Asilomar, California last year

and produced this list of 23 principles

which have since been signed by over 1,000 AI researchers

and key industry leaders,

and I want to tell you about three of these principles.

One is that we should avoid an arms race and lethal autonomous weapons.

The idea here is that any science can be used for new ways of helping people

or new ways of harming people.

For example, biology and chemistry are much more likely to be used

for new medicines or new cures than for new ways of killing people,

because biologists and chemists pushed hard --

and successfully --

for bans on biological and chemical weapons.

And in the same spirit,

most AI researchers want to stigmatize and ban lethal autonomous weapons.

Another Asilomar AI principle

is that we should mitigate AI-fueled income inequality.

I think that if we can grow the economic pie dramatically with AI

and we still can't figure out how to divide this pie

so that everyone is better off,

then shame on us.

(Applause)

Alright, now raise your hand if your computer has ever crashed.

(Laughter)

Wow, that's a lot of hands.

Well, then you'll appreciate this principle

that we should invest much more in AI safety research,

because as we put AI in charge of even more decisions and infrastructure,

we need to figure out how to transform today's buggy and hackable computers

into robust AI systems that we can really trust,

because otherwise,

all this awesome new technology can malfunction and harm us,

or get hacked and be turned against us.

And this AI safety work has to include work on AI value alignment,

because the real threat from AGI isn't malice,

like in silly Hollywood movies,

but competence --

AGI accomplishing goals that just aren't aligned with ours.

For example, when we humans drove the West African black rhino extinct,

we didn't do it because we were a bunch of evil rhinoceros haters, did we?

We did it because we were smarter than them

and our goals weren't aligned with theirs.

But AGI is by definition smarter than us,

so to make sure that we don't put ourselves in the position of those rhinos

if we create AGI,

we need to figure out how to make machines understand our goals,

adopt our goals and retain our goals.

And whose goals should these be, anyway?

Which goals should they be?

This brings us to the third part of our rocket metaphor: the destination.

We're making AI more powerful,

trying to figure out how to steer it,

but where do we want to go with it?

This is the elephant in the room that almost nobody talks about --

not even here at TED --

because we're so fixated on short-term AI challenges.

Look, our species is trying to build AGI,

motivated by curiosity and economics,

but what sort of future society are we hoping for if we succeed?

We did an opinion poll on this recently,

and I was struck to see

that most people actually want us to build superintelligence:

AI that's vastly smarter than us in all ways.

What there was the greatest agreement on was that we should be ambitious

and help life spread into the cosmos,

but there was much less agreement about who or what should be in charge.

And I was actually quite amused

to see that there's some some people who want it to be just machines.

(Laughter)

And there was total disagreement about what the role of humans should be,

even at the most basic level,

so let's take a closer look at possible futures

that we might choose to steer toward, alright?

So don't get me wrong here.

I'm not talking about space travel,

merely about humanity's metaphorical journey into the future.

So one option that some of my AI colleagues like

is to build superintelligence and keep it under human control,

like an enslaved god,

disconnected from the internet

and used to create unimaginable technology and wealth

for whoever controls it.

But Lord Acton warned us

that power corrupts, and absolute power corrupts absolutely,

so you might worry that maybe we humans just aren't smart enough,

or wise enough rather,

to handle this much power.

Also, aside from any moral qualms you might have

about enslaving superior minds,

you might worry that maybe the superintelligence could outsmart us,

break out and take over.

But I also have colleagues who are fine with AI taking over

and even causing human extinction,

as long as we feel the the AIs are our worthy descendants,

like our children.

But how would we know that the AIs have adopted our best values

and aren't just unconscious zombies tricking us into anthropomorphizing them?

Also, shouldn't those people who don't want human extinction

have a say in the matter, too?

Now, if you didn't like either of those two high-tech options,

it's important to remember that low-tech is suicide

from a cosmic perspective,

because if we don't go far beyond today's technology,

the question isn't whether humanity is going to go extinct,

merely whether we're going to get taken out

by the next killer asteroid, supervolcano

or some other problem that better technology could have solved.

So, how about having our cake and eating it ...

with AGI that's not enslaved

but treats us well because its values are aligned with ours?

This is the gist of what Eliezer Yudkowsky has called "friendly AI,"

and if we can do this, it could be awesome.

It could not only eliminate negative experiences like disease, poverty,

crime and other suffering,

but it could also give us the freedom to choose

from a fantastic new diversity of positive experiences --

basically making us the masters of our own destiny.

So in summary,

our situation with technology is complicated,

but the big picture is rather simple.

Most AI researchers expect AGI within decades,

and if we just bumble into this unprepared,

it will probably be the biggest mistake in human history --

let's face it.

It could enable brutal, global dictatorship

with unprecedented inequality, surveillance and suffering,

and maybe even human extinction.

But if we steer carefully,

we could end up in a fantastic future where everybody's better off:

the poor are richer, the rich are richer,

everybody is healthy and free to live out their dreams.

Now, hang on.

Do you folks want the future that's politically right or left?

Do you want the pious society with strict moral rules,

or do you an hedonistic free-for-all,

more like Burning Man 24/7?

Do you want beautiful beaches, forests and lakes,

or would you prefer to rearrange some of those atoms with the computers,

enabling virtual experiences?

With friendly AI, we could simply build all of these societies

and give people the freedom to choose which one they want to live in

because we would no longer be limited by our intelligence,

merely by the laws of physics.

So the resources and space for this would be astronomical --

literally.

So here's our choice.

We can either be complacent about our future,

taking as an article of blind faith

that any new technology is guaranteed to be beneficial,

and just repeat that to ourselves as a mantra over and over and over again

as we drift like a rudderless ship towards our own obsolescence.

Or we can be ambitious --

thinking hard about how to steer our technology

and where we want to go with it

to create the age of amazement.

We're all here to celebrate the age of amazement,

and I feel that its essence should lie in becoming not overpowered

but empowered by our technology.

Thank you.

(Applause)