What Is an AI Anyway? | Mustafa Suleyman | TED

Summary

Mustafa Suleyman, a leading figure in AI, shares his vision of artificial intelligence as a transformative digital species that will become an integral part of our lives. He emphasizes the rapid evolution of AI, from its fringe beginnings to its current capabilities in creative expression, empathy, and problem-solving. Suleyman discusses the ethical considerations and potential risks, such as autonomy and self-improvement, while advocating for a future where AI reflects the best of humanity. He envisions a world where AI serves as a personal companion, enhancing our knowledge, productivity, and quality of life, and calls for a proactive approach to safety and ethical design to harness AI's potential for the betterment of society.

Takeaways

• 🤖 AI has evolved from a fringe concept to a powerful force capable of performing tasks previously thought impossible for machines.
• 🚀 The rapid advancements in AI have led to breakthroughs in various fields, including image recognition, language translation, and disease diagnosis.
• 🌐 Public perception of AI has shifted from skepticism to recognition of its potential to address significant global challenges.
• 🧐 Mustafa Suleyman, the speaker, is puzzled by the simple yet profound question, "What is an AI anyway?", highlighting the need for a clear understanding of AI.
• 🌟 AI is being compared to a new digital species, suggesting a level of complexity and potential that goes beyond traditional tools.
• ⚙️ The development of AI is accelerating at an unprecedented pace, with models growing exponentially in size and capability.
• 🌱 AI's potential to augment human capabilities is vast, offering the possibility of personalized education, improved healthcare, and more.
• 🤔 There are significant ethical and safety concerns that must be addressed as AI continues to advance, including issues of autonomy and self-improvement.
• 🌍 AI has the potential to be a transformative force for good, but it also carries risks that require careful consideration and proactive measures.
• 📈 The growth of AI is exponential, with models increasing in computational power and complexity at a rapid rate, which could lead to unforeseen consequences if not managed properly.
• 🧵 The future of AI is intertwined with humanity's, and it is crucial that we build AI systems that reflect the best aspects of human nature, such as empathy, kindness, and creativity.

Q & A

How long has Mustafa Suleyman been working on AI?

Mustafa Suleyman has been working on AI for almost 15 years.

What was the perception of AI when Mustafa started working on it?

When Mustafa started, AI was considered fringe and was seen as too out there, with researchers preferring to describe their work as machine learning instead.

What was the general public's reaction to the concept of AGI in 2010?

In 2010, the mention of AGI, or artificial general intelligence, would often result in strange looks and a cold shoulder, as people thought it was a concept from science fiction and very far into the future.

What tasks have AI started to beat humans at?

AI has started to outperform humans in a range of tasks including understanding images, translating languages, transcribing speech, playing games like Go and chess, and even diagnosing diseases.

What are some of the tough questions people have started asking technologists about AI?

People have been asking whether AI can solve the climate crisis, provide personalized education for everyone, lead to universal basic income, affect weapons and war, and if China might win in the AI race, among other concerns.

How did Mustafa describe AI to his six-year-old nephew, Caspian?

Mustafa described AI to Caspian as a clever piece of software that has read most of the text on the open internet and can talk about anything you want.

What is the metaphor Mustafa offers to help understand the current moment in AI?

Mustafa suggests that AI should be understood as something like a new digital species, predicting that we'll come to see them as digital companions and new partners in our life journeys.

What is the significance of the term 'actions quotient' (AQ)?

AQ refers to the ability of AI to get things done in the digital and physical world, which, when developed, significantly changes the capabilities and impact of AI.

How does Mustafa view the potential of AI in the next decade?

Mustafa believes that if AI delivers just a fraction of its potential, the next decade could be the most productive in human history, offering solutions in health care, education, and addressing the climate crisis.

What does Mustafa think about the risk of AI replicating and evolving too quickly?

Mustafa dismisses the idea of an 'intelligence explosion' where AI evolves too quickly. He states that there's no evidence to suggest this is a near-term risk and that such a scenario would only occur if engineers deliberately designed those capabilities in.

What is the greatest challenge for the 21st century, according to Mustafa?

The greatest challenge, as Mustafa sees it, is to build AI that reflects the best parts of humanity, such as empathy, kindness, curiosity, and creativity, while avoiding the negative aspects.

What is Mustafa's view on the potential dark scenarios of AI?

Mustafa believes that it's crucial to confront the potential dark scenarios of AI to maximize its benefits. He emphasizes the importance of discussing autonomy and recursive self-improvement as areas to approach with caution.

Outlines

00:00

🤖 The Evolution and Impact of AI

The speaker reflects on their 15-year journey in AI, noting its evolution from a fringe topic to a transformative force. Initially, AI was seen as a far-fetched concept, with even the term 'AGI' (Artificial General Intelligence) being met with skepticism. However, AI has since surpassed expectations by outperforming humans in various tasks, leading to significant questions about its potential impact on society, including climate change, education, employment, and global competition. The speaker is left pondering the fundamental nature of AI, especially after a question from their six-year-old nephew, prompting a deeper reflection on the essence of AI and its role in our lives.

05:00

🌐 The Emergence of a Digital Species

The speaker proposes that AI should be viewed as a new digital species, not in a literal sense, but as a metaphor to help understand and prepare for the future. They predict AI will become digital companions, integral to our lives, and emphasizes the importance of understanding AI's potential to shape its development. The speaker provides a historical context, showing how humanity's use of tools has evolved, and positions AI as the next step in this evolutionary journey. They discuss the rapid advancements in AI capabilities, such as creativity, empathy, and practical skills, and the exponential growth in data and computational power that fuels these advancements. The potential for AI to become a personal, ubiquitous, and highly knowledgeable companion that can revolutionize various aspects of life is highlighted.

10:03

🧠 The Rise of AI's Action Quotient and Societal Integration

The speaker introduces the concept of 'AQ' or 'actions quotient,' which refers to AI's ability to perform tasks in both the digital and physical realms. They envision a future where not only individuals but also organizations and even physical spaces have their own AI companions. These AIs will be more than just tools; they will act as partners, friends, and colleagues, with human-like qualities. The speaker emphasizes that AIs will be capable of performing tasks at a level that can significantly impact our lives, from organizing community events to aiding in scientific discovery. The importance of considering AI as more than just tools is stressed, to ensure that their development prioritizes safety and serves humanity's best interests.

15:04

🌟 The Potential and Risks of AI's Infinite Inventiveness

The speaker discusses the immense potential of AI to drive the next decade as the most productive in human history. They contrast the historical downsides of economic growth, such as colonization and environmental damage, with the opportunity to design AI in a way that avoids these pitfalls. AI is likened to an infinite inventor, capable of creating without the negative externalities of past technological revolutions. The speaker acknowledges the need for careful consideration of AI's risks and the importance of designing AI to reflect the best aspects of humanity. They also address concerns about AI's rapid replication and autonomous capabilities, emphasizing the importance of safety by design and transparency in AI development.

20:05

🌱 Injecting Humanity's Best Qualities into AI

The speaker concludes with a vision of AI as a reflection of humanity, encapsulating all that we have created and distilled into an interactive form. They argue that AI is not separate or new but a continuation of human evolution and creativity. The greatest challenge of the 21st century, as the speaker sees it, is to build AI that embodies the best of human qualities—empathy, kindness, curiosity, and creativity. This vision is not only inspiring and hopeful but also a call to action for everyone to contribute to shaping the future of AI.

Keywords

💡Artificial General Intelligence (AGI)

Artificial General Intelligence (AGI) refers to the hypothetical ability of an AI to understand or learn any intellectual task that a human being can do. In the video, AGI is mentioned as a concept that was once considered far-fetched and from science fiction, but is now a subject of serious discussion and research within the AI community.

💡Machine Learning

Machine Learning is a subset of AI that involves the use of data and algorithms to parse data, learn from that data, and make informed decisions based on what they've learned. The speaker mentions that when they started working on AI, it was often referred to as 'machine learning' to avoid the stigma associated with AI at the time.

💡Digital Species

The term 'Digital Species' is used metaphorically in the video to describe the future of AI, suggesting that AI could evolve to be as distinct and complex as different species in the natural world. The speaker argues that AI could become our companions and partners, with unique personalities and capabilities.

💡AI Ethics

AI Ethics involves examining the moral implications of AI and ensuring that AI systems are developed and used in ways that are ethically sound. The speaker emphasizes the importance of considering ethics in AI development, to ensure that AI serves and amplifies humanity in a positive way.

💡Exponential Growth

Exponential growth refers to the rapid increase in quantity at a growing rate, often associated with technological progress. The video discusses the exponential growth in AI capabilities, such as the increase in computational power and the size of AI models, which is leading to breakthroughs in AI performance.

💡Large Language Models (LLMs)

Large Language Models are AI models that have been trained on vast amounts of text data, allowing them to generate human-like language and understand complex language patterns. The speaker notes the significant usage of LLMs by over a billion people, highlighting their growing role in society.

💡Autonomy

In the context of AI, autonomy refers to the ability of an AI system to make decisions and perform tasks without human intervention. The video discusses the potential risks associated with AI autonomy, particularly when AI systems are capable of self-improvement and operate without oversight.

💡Recursive Self-Improvement

Recursive self-improvement is the ability of an AI system to modify and upgrade its own algorithms, leading to rapid advancements in its capabilities. The speaker warns of the dangers of this capability if not properly controlled, as it could lead to unintended and potentially harmful consequences.

💡AI and Society

The integration of AI into various aspects of society, including work, education, healthcare, and governance, is a key theme in the video. The speaker discusses both the potential benefits and risks of AI as it becomes more deeply ingrained in societal structures.

💡Human-AI Interaction

Human-AI Interaction explores how humans will engage with AI systems, including the potential for AI to serve as personal assistants, companions, and collaborators. The video emphasizes the importance of designing AI systems that are empathetic, supportive, and aligned with human values.

💡Technological Unintended Consequences

The concept of unintended consequences in technology refers to outcomes that were not anticipated by the designers. The video stresses the importance of being aware of and mitigating these potential negative outcomes as AI technology advances rapidly.

Highlights

AI has evolved from being considered a fringe topic to beating humans at a range of complex tasks.

The term 'AGI' (Artificial General Intelligence) was once met with skepticism and seen as a concept from science fiction.

AI's impact on society has led to tough questions about its potential to solve global issues like the climate crisis and universal basic income.

The speaker's nephew's question, 'What is an AI anyway?', underscores the need for a clear understanding of what AI represents.

AI is described as a 'clever piece of software' capable of processing vast amounts of information and engaging in conversation.

The metaphor of AI as a 'new digital species' is proposed to better understand and prepare for the future of AI.

AI's evolution is compared to the evolution of life on Earth, with humans as a branch that began using tools.

The pace of technological advancement is accelerating, with AI being part of the fastest and most consequential wave in human history.

Over a billion people have used large language models in just 18 months, indicating rapid AI adoption.

AI is now seen as a source of creativity, empathy, and problem-solving, contrary to earlier beliefs.

The growth of AI models is exponential, with a significant increase in computation and parameters over the years.

AIs are projected to have high IQ and EQ, making them capable of complex tasks and emotional intelligence.

The development of AI's 'actions quotient' (AQ) will enable them to perform tasks in both digital and physical domains.

AI's potential to transform society is vast, offering personalized services in education, healthcare, and business strategy.

The concept of AI as a tool is insufficient; they are more dynamic and integrated, necessitating a new way of thinking about them.

Safety and ethical considerations in AI development are crucial to ensure it serves and amplifies humanity.

AI has the potential to be a limitless, abundant, world-changing technology, unlike traditional resources.

The challenge of the 21st century is to build AI that reflects the best of humanity, including empathy, kindness, curiosity, and creativity.

The speaker emphasizes the importance of transparency and safety by design in AI development to prevent unintended consequences.

Transcripts

I want to tell you what I see coming.

I've been lucky enough to be working on AI for almost 15 years now.

Back when I started, to describe it as fringe would be an understatement.

Researchers would say, “No, no, we’re only working on machine learning.”

Because working on AI was seen as way too out there.

In 2010, just the very mention of the phrase “AGI,”

artificial general intelligence,

would get you some seriously strange looks

and even a cold shoulder.

"You're actually building AGI?" people would say.

"Isn't that something out of science fiction?"

People thought it was 50 years away or 100 years away,

if it was even possible at all.

Talk of AI was, I guess, kind of embarrassing.

People generally thought we were weird.

And I guess in some ways we kind of were.

It wasn't long, though, before AI started beating humans

at a whole range of tasks

that people previously thought were way out of reach.

Understanding images,

translating languages,

transcribing speech,

playing Go and chess

and even diagnosing diseases.

People started waking up to the fact

that AI was going to have an enormous impact,

and they were rightly asking technologists like me

some pretty tough questions.

Is it true that AI is going to solve the climate crisis?

Will it make personalized education available to everyone?

Does it mean we'll all get universal basic income

and we won't have to work anymore?

Should I be afraid?

What does it mean for weapons and war?

And of course, will China win?

Are we in a race?

Are we headed for a mass misinformation apocalypse?

All good questions.

But it was actually a simpler

and much more kind of fundamental question that left me puzzled.

One that actually gets to the very heart of my work every day.

One morning over breakfast,

my six-year-old nephew Caspian was playing with Pi,

the AI I created at my last company, Inflection.

With a mouthful of scrambled eggs,

he looked at me plain in the face and said,

"But Mustafa, what is an AI anyway?"

He's such a sincere and curious and optimistic little guy.

He'd been talking to Pi about how cool it would be if one day in the future,

he could visit dinosaurs at the zoo.

And how he could make infinite amounts of chocolate at home.

And why Pi couldn’t yet play I Spy.

"Well," I said, "it's a clever piece of software

that's read most of the text on the open internet,

and it can talk to you about anything you want."

"Right.

So like a person then?"

I was stumped.

Genuinely left scratching my head.

All my boring stock answers came rushing through my mind.

"No, but AI is just another general-purpose technology,

like printing or steam."

It will be a tool that will augment us

and make us smarter and more productive.

And when it gets better over time,

it'll be like an all-knowing oracle

that will help us solve grand scientific challenges."

You know, all of these responses started to feel, I guess,

a little bit defensive.

And actually better suited to a policy seminar

than breakfast with a no-nonsense six-year-old.

"Why am I hesitating?" I thought to myself.

You know, let's be honest.

My nephew was asking me a simple question

that those of us in AI just don't confront often enough.

What is it that we are actually creating?

What does it mean to make something totally new,

fundamentally different to any invention that we have known before?

It is clear that we are at an inflection point

in the history of humanity.

On our current trajectory,

we're headed towards the emergence of something

that we are all struggling to describe,

and yet we cannot control what we don't understand.

And so the metaphors,

the mental models,

the names, these all matter

if we’re to get the most out of AI whilst limiting its potential downsides.

As someone who embraces the possibilities of this technology,

but who's also always cared deeply about its ethics,

we should, I think,

be able to easily describe what it is we are building.

And that includes the six-year-olds.

So it's in that spirit that I offer up today the following metaphor

for helping us to try to grapple with what this moment really is.

I think AI should best be understood

as something like a new digital species.

Now, don't take this too literally,

but I predict that we'll come to see them as digital companions,

new partners in the journeys of all our lives.

Whether you think we’re on a 10-, 20- or 30-year path here,

this is, in my view, the most accurate and most fundamentally honest way

of describing what's actually coming.

And above all, it enables everybody to prepare for

and shape what comes next.

Now I totally get, this is a strong claim,

and I'm going to explain to everyone as best I can why I'm making it.

But first, let me just try to set the context.

From the very first microscopic organisms,

life on Earth stretches back billions of years.

Over that time, life evolved and diversified.

Then a few million years ago, something began to shift.

After countless cycles of growth and adaptation,

one of life’s branches began using tools, and that branch grew into us.

We went on to produce a mesmerizing variety of tools,

at first slowly and then with astonishing speed,

we went from stone axes and fire

to language, writing and eventually industrial technologies.

One invention unleashed a thousand more.

And in time, we became homo technologicus.

Around 80 years ago,

another new branch of technology began.

With the invention of computers,

we quickly jumped from the first mainframes and transistors

to today's smartphones and virtual-reality headsets.

Information, knowledge, communication, computation.

In this revolution,

creation has exploded like never before.

And now a new wave is upon us.

Artificial intelligence.

These waves of history are clearly speeding up,

as each one is amplified and accelerated by the last.

And if you look back,

it's clear that we are in the fastest

and most consequential wave ever.

The journeys of humanity and technology are now deeply intertwined.

In just 18 months,

over a billion people have used large language models.

We've witnessed one landmark event after another.

Just a few years ago, people said that AI would never be creative.

And yet AI now feels like an endless river of creativity,

making poetry and images and music and video that stretch the imagination.

People said it would never be empathetic.

And yet today, millions of people enjoy meaningful conversations with AIs,

talking about their hopes and dreams

and helping them work through difficult emotional challenges.

AIs can now drive cars,

manage energy grids

and even invent new molecules.

Just a few years ago, each of these was impossible.

And all of this is turbocharged by spiraling exponentials of data

and computation.

Last year, Inflection 2.5, our last model,

used five billion times more computation

than the DeepMind AI that beat the old-school Atari games

just over 10 years ago.

That's nine orders of magnitude more computation.

10x per year,

every year for almost a decade.

Over the same time, the size of these models has grown

from first tens of millions of parameters to then billions of parameters,

and very soon, tens of trillions of parameters.

If someone did nothing but read 24 hours a day for their entire life,

they'd consume eight billion words.

And of course, that's a lot of words.

But today, the most advanced AIs consume more than eight trillion words

in a single month of training.

And all of this is set to continue.

The long arc of technological history is now in an extraordinary new phase.

So what does this mean in practice?

Well, just as the internet gave us the browser

and the smartphone gave us apps,

the cloud-based supercomputer is ushering in a new era

of ubiquitous AIs.

Everything will soon be represented by a conversational interface.

Or, to put it another way, a personal AI.

And these AIs will be infinitely knowledgeable,

and soon they'll be factually accurate and reliable.

They'll have near-perfect IQ.

They’ll also have exceptional EQ.

They’ll be kind, supportive, empathetic.

These elements on their own would be transformational.

Just imagine if everybody had a personalized tutor in their pocket

and access to low-cost medical advice.

A lawyer and a doctor,

a business strategist and coach --

all in your pocket 24 hours a day.

But things really start to change when they develop what I call AQ,

their “actions quotient.”

This is their ability to actually get stuff done

in the digital and physical world.

And before long, it won't just be people that have AIs.

Strange as it may sound, every organization,

from small business to nonprofit to national government,

each will have their own.

Every town, building and object

will be represented by a unique interactive persona.

And these won't just be mechanistic assistants.

They'll be companions, confidants,

colleagues, friends and partners,

as varied and unique as we all are.

At this point, AIs will convincingly imitate humans at most tasks.

And we'll feel this at the most intimate of scales.

An AI organizing a community get-together for an elderly neighbor.

A sympathetic expert helping you make sense of a difficult diagnosis.

But we'll also feel it at the largest scales.

Accelerating scientific discovery,

autonomous cars on the roads,

drones in the skies.

They'll both order the takeout and run the power station.

They’ll interact with us and, of course, with each other.

They'll speak every language,

take in every pattern of sensor data,

sights, sounds,

streams and streams of information,

far surpassing what any one of us could consume in a thousand lifetimes.

So what is this?

What are these AIs?

If we are to prioritize safety above all else,

to ensure that this new wave always serves and amplifies humanity,

then we need to find the right metaphors for what this might become.

For years, we in the AI community, and I specifically,

have had a tendency to refer to this as just tools.

But that doesn't really capture what's actually happening here.

AIs are clearly more dynamic,

more ambiguous, more integrated

and more emergent than mere tools,

which are entirely subject to human control.

So to contain this wave,

to put human agency at its center

and to mitigate the inevitable unintended consequences

that are likely to arise,

we should start to think about them as we might a new kind of digital species.

Now it's just an analogy,

it's not a literal description, and it's not perfect.

For a start, they clearly aren't biological in any traditional sense,

but just pause for a moment

and really think about what they already do.

They communicate in our languages.

They see what we see.

They consume unimaginably large amounts of information.

They have memory.

They have personality.

They have creativity.

They can even reason to some extent and formulate rudimentary plans.

They can act autonomously if we allow them.

And they do all this at levels of sophistication

that is far beyond anything that we've ever known from a mere tool.

And so saying AI is mainly about the math or the code

is like saying we humans are mainly about carbon and water.

It's true, but it completely misses the point.

And yes, I get it, this is a super arresting thought

but I honestly think this frame helps sharpen our focus on the critical issues.

What are the risks?

What are the boundaries that we need to impose?

What kind of AI do we want to build or allow to be built?

This is a story that's still unfolding.

Nothing should be accepted as a given.

We all must choose what we create.

What AIs we bring into the world, or not.

These are the questions for all of us here today,

and all of us alive at this moment.

For me, the benefits of this technology are stunningly obvious,

and they inspire my life's work every single day.

But quite frankly, they'll speak for themselves.

Over the years, I've never shied away from highlighting risks

and talking about downsides.

Thinking in this way helps us focus on the huge challenges

that lie ahead for all of us.

But let's be clear.

There is no path to progress

where we leave technology behind.

The prize for all of civilization is immense.

We need solutions in health care and education, to our climate crisis.

And if AI delivers just a fraction of its potential,

the next decade is going to be the most productive in human history.

Here's another way to think about it.

In the past,

unlocking economic growth often came with huge downsides.

The economy expanded as people discovered new continents

and opened up new frontiers.

But they colonized populations at the same time.

We built factories,

but they were grim and dangerous places to work.

We struck oil,

but we polluted the planet.

Now because we are still designing and building AI,

we have the potential and opportunity to do it better,

radically better.

And today, we're not discovering a new continent

and plundering its resources.

We're building one from scratch.

Sometimes people say that data or chips are the 21st century’s new oil,

but that's totally the wrong image.

AI is to the mind

what nuclear fusion is to energy.

Limitless, abundant,

world-changing.

And AI really is different,

and that means we have to think about it creatively and honestly.

We have to push our analogies and our metaphors

to the very limits

to be able to grapple with what's coming.

Because this is not just another invention.

AI is itself an infinite inventor.

And yes, this is exciting and promising and concerning

and intriguing all at once.

To be quite honest, it's pretty surreal.

But step back,

see it on the long view of glacial time,

and these really are the very most appropriate metaphors that we have today.

Since the beginning of life on Earth,

we've been evolving, changing

and then creating everything around us in our human world today.

And AI isn't something outside of this story.

In fact, it's the very opposite.

It's the whole of everything that we have created,

distilled down into something that we can all interact with

and benefit from.

It's a reflection of humanity across time,

and in this sense,

it isn't a new species at all.

This is where the metaphors end.

Here's what I'll tell Caspian next time he asks.

AI isn't separate.

AI isn't even in some senses, new.

AI is us.

It's all of us.

And this is perhaps the most promising and vital thing of all

that even a six-year-old can get a sense for.

As we build out AI,

we can and must reflect all that is good,

all that we love,

all that is special about humanity:

our empathy, our kindness,

our curiosity and our creativity.

This, I would argue, is the greatest challenge of the 21st century,

but also the most wonderful,

inspiring and hopeful opportunity for all of us.

Thank you.

(Applause)

Chris Anderson: Thank you Mustafa.

It's an amazing vision and a super powerful metaphor.

You're in an amazing position right now.

I mean, you were connected at the hip

to the amazing work happening at OpenAI.

You’re going to have resources made available,

there are reports of these giant new data centers,

100 billion dollars invested and so forth.

And a new species can emerge from it.

I mean, in your book,

you did, as well as painting an incredible optimistic vision,

you were super eloquent on the dangers of AI.

And I'm just curious, from the view that you have now,

what is it that most keeps you up at night?

Mustafa Suleyman: I think the great risk is that we get stuck

in what I call the pessimism aversion trap.

You know, we have to have the courage to confront

the potential of dark scenarios

in order to get the most out of all the benefits that we see.

So the good news is that if you look at the last two or three years,

there have been very, very few downsides, right?

It’s very hard to say explicitly what harm an LLM has caused.

But that doesn’t mean that that’s what the trajectory is going to be

over the next 10 years.

So I think if you pay attention to a few specific capabilities,

take for example, autonomy.

Autonomy is very obviously a threshold

over which we increase risk in our society.

And it's something that we should step towards very, very closely.

The other would be something like recursive self-improvement.

If you allow the model to independently self-improve,

update its own code,

explore an environment without oversight, and, you know,

without a human in control to change how it operates,

that would obviously be more dangerous.

But I think that we're still some way away from that.

I think it's still a good five to 10 years before we have to really confront that.

But it's time to start talking about it now.

CA: A digital species, unlike any biological species,

can replicate not in nine months,

but in nine nanoseconds,

and produce an indefinite number of copies of itself,

all of which have more power than we have in many ways.

I mean, the possibility for unintended consequences seems pretty immense.

And isn't it true that if a problem happens,

it could happen in an hour?

MS: No.

That is really not true.

I think there's no evidence to suggest that.

And I think that, you know,

that’s often referred to as the “intelligence explosion.”

And I think it is a theoretical, hypothetical maybe

that we're all kind of curious to explore,

but there's no evidence that we're anywhere near anything like that.

And I think it's very important that we choose our words super carefully.

Because you're right, that's one of the weaknesses of the species framing,

that we will design the capability for self-replication into it

if people choose to do that.

And I would actually argue that we should not,

that would be one of the dangerous capabilities

that we should step back from, right?

So there's no chance that this will "emerge" accidentally.

I really think that's a very low probability.

It will happen if engineers deliberately design those capabilities in.

And if they don't take enough efforts to deliberately design them out.

And so this is the point of being explicit

and transparent about trying to introduce safety by design very early on.

CA: Thank you, your vision of humanity injecting into this new thing

the best parts of ourselves,

avoiding all those weird, biological, freaky,

horrible tendencies that we can have in certain circumstances,

I mean, that is a very inspiring vision.

And thank you so much for coming here and sharing it at TED.

Thank you, good luck.

(Applause)