Mustafa Suleyman & Yuval Noah Harari -FULL DEBATE- What does the AI revolution mean for our future?
Summary
TLDRIn a thought-provoking discussion, historian Yuval Noah Harari and entrepreneur Mustafa Suleiman, co-founder and CEO of Inflection AI, debate the future implications of AI. Suleiman, a key figure in the AI revolution, envisions a future where AI could perform complex tasks like creating new products and managing investments, potentially within the next five years. Harari, however, expresses concerns about the shift of power from humans to AI, suggesting it could mark the end of human-dominated history. They explore the potential benefits of AI, such as advancements in healthcare and solving global issues like climate change, but also delve into the risks, including job displacement, political disruptions, and the erosion of trust in democratic processes due to AI's ability to generate convincing but false information. The conversation underscores the need for careful governance, precautionary measures, and the development of new institutions to manage the rapid advancements in AI technology, ensuring that the technology serves humanity without causing irreversible damage.
Takeaways
- đ Mustafa Suleiman, co-founder and CEO of Inflection AI, envisions AI systems within five years that could potentially manage complex tasks like creating a new product from scratch, including market research, manufacturing, and sales.
- đ Yuval Noah Harari, a historian and author, suggests that AI's ability to make independent decisions and create new ideas could mark the end of human-dominated history, with control shifting to non-human intelligence.
- đ Suleiman highlights the potential upsides of AI, such as transformative improvements in health, accelerated innovation, and addressing global challenges like climate change.
- đ€ Harari expresses concern that the positive potential of AI might not be worth the risks, especially considering humanity's track record as the most intelligent but also destructive entity on Earth.
- đ The discussion touches on the impact of AI on jobs, suggesting that while AI might not destroy all jobs, it could cause significant disruptions and require careful management of transitions in the job market.
- đ There's a concern that AI could destabilize liberal democracy by eroding trust and the ability for large-scale political conversations, which are foundational to its functioning.
- đĄ Suleiman proposes a modern Turing test to evaluate AI's capabilities, involving tasks like creating a new product and managing the entire process autonomously.
- đĄ The conversation emphasizes the need for regulation and governance of AI, including the precautionary principle and the establishment of international investigatory powers to assess AI risks.
- đ Red teaming, or adversarial testing, is suggested as a method to identify and address weaknesses in AI systems before they can be exploited.
- đ« There's a call for certain capabilities, like autonomy and recursive self-improvement, to be considered high-risk and potentially off-limits to prevent unforeseen consequences.
- đ± Harari suggests investing in human consciousness and mind development alongside AI, to ensure that humanity can keep pace with and manage the artificial intelligence it creates.
Q & A
What are Yuval Noah Harari's concerns about the future of AI according to the transcript?
-Yuval Noah Harari expresses concern that AI technology might reach a point where it can make decisions independently and create new ideas without human input. He fears this could end human-dominated history, shifting control from humans to AI, potentially leading to significant societal and ethical consequences.
What potential benefits of AI does Mustafa Suleiman discuss in the transcript?
-Mustafa Suleiman highlights several potential benefits of AI, including dramatic improvements in human health, acceleration of scientific discovery, and solving major global challenges like climate change. He envisions AI augmenting human capabilities, making people more efficient, creative, and capable.
According to the discussion, what does the future hold for AI in terms of employment?
-The future of AI in employment is debated, with Mustafa Suleiman suggesting that AI might not pose a significant threat to jobs in the short term (10-20 years), but it could in the longer term (30-50 years). Harari adds that while total job elimination is unlikely, the transition could be disruptive, with certain jobs disappearing and others appearing, potentially causing significant economic and social shifts.
How does Yuval Noah Harari link the concept of democracy to information technology?
-Harari explains that modern democracy, which allows for widespread, large-scale political participation and conversation, relies heavily on information technology such as newspapers, radio, and TV. He suggests that as these technologies evolve, the structure of democracy may need to adapt if these new technologies change how people communicate and trust information.
What risks does the widespread use of AI pose to liberal democracy, according to the transcript?
-The transcript discusses the risk of AI flooding the online space with non-human entities that can impersonate humans, potentially leading to a breakdown in trust among the populace. This could hinder meaningful political conversations and destabilize the democratic process, making it difficult to discern true information from manipulation.
What are Mustafa Suleiman's views on AI's potential impact on political systems?
-Mustafa Suleiman notes the importance of maintaining human oversight and establishing strong governance to prevent AI from being used unethically in political systems, such as in elections or for creating counterfeit digital identities. He emphasizes the need for proactive measures to safeguard democratic processes.
What does Yuval Noah Harari mean by AI 'hacking the operating system' of liberal democracy?
-Harari uses this metaphor to describe how AI could undermine the foundational aspects of liberal democracy by eroding trust and facilitating impersonation and deception at scale. This could destabilize the very mechanisms that allow democracies to function, such as free and fair elections and informed public discourse.
How does Mustafa Suleiman propose to manage the risks associated with AI?
-Suleiman suggests a proactive approach, including red teaming AI systems to test their responses in extreme scenarios, establishing shared safety standards among AI developers, and creating restrictions on certain high-risk AI capabilities. He advocates for a precautionary principle to manage AI development responsibly.
Outlines
đ€ Introduction to the AI Debate
Historian Yuval Noah Harari and entrepreneur Mustafa Suleiman join a discussion on the implications of AI on the future, including its impact on employment, geopolitics, and the survival of liberal democracy. Harari, a best-selling author, and Suleiman, co-founder of DeepMind and Inflection AI, bring different perspectives to the debate. Suleiman paints a picture of the future where AI could have human-level capabilities within five years, highlighting the generative AI revolution's advancements.
đ The Future of AI and Its Capabilities
Suleiman discusses the potential for AI to perform complex tasks autonomously, such as creating a new product within a few months. He proposes a modern Turing test to evaluate AI's capabilities. Harari expresses concern that AI could mark the end of human-dominated history, as it could make independent decisions and create new ideas. Suleiman counters with optimism about AI's potential to address difficult problems and create positive outcomes.
đ The Impact of AI on Jobs and Society
The conversation turns to the impact of AI on jobs. Harari raises the concern of social and political disruptions due to unemployment caused by AI. Suleiman acknowledges the potential risks but emphasizes the need for careful development and governance of AI. They discuss the historical context of job transitions and the importance of managing the transition period抄ćć°.
đ The Risks to Political Systems and Democracy
The discussion moves to the potential risks AI poses to political systems. Harari warns that the rise of AI could lead to a collapse in trust and the breakdown of democratic conversation. Suleiman agrees on the need for regulation and governance to prevent misuse, such as the impersonation of digital people. They both emphasize the importance of technical and governance mechanisms to maintain trust in political conversations.
đ Containment and Regulation of AI
Suleiman and Harari discuss the challenges of containing and regulating AI, especially when it comes to mass proliferation and the power of nation-states. Suleiman suggests self-organizing initiatives and a precautionary principle, while Harari calls for new institutions capable of understanding and reacting to fast technological developments. They agree on the necessity of international cooperation and the importance of values in shaping the future of AI.
đ The Unpredictability and Potential of AI
Harari and Suleiman reflect on the unpredictability of AI and its potential to act in unanticipated ways. Suleiman advocates for a cautious approach, with certain high-risk capabilities being taken off the table. Harari emphasizes the need for investment in human development alongside AI. Both agree on the importance of setting social and political norms for AI use and the challenges of establishing regulations in a rapidly evolving field.
đ Conclusion and Commitment to AI's Positive Development
In conclusion, Suleiman expresses his commitment to creating AI that adheres to strict safety constraints, believing in the potential for positive outcomes. Harari, while cautious, acknowledges the inevitability of AI's development and the importance of investing in human consciousness. The discussion highlights the need for ongoing dialogue, regulation, and ethical considerations in the face of AI's rapid advancement.
Mindmap
Keywords
đĄAI Revolution
đĄHuman-level AI
đĄGenerative AI
đĄTuring Test
đĄ
đĄAutonomous AI
đĄAI and Jobs
đĄLiberal Democracy
đĄInformation Technology
đĄRed Teaming
đĄPrecautionary Principle
đĄGeopolitics
Highlights
Yuval Noah Harari and Mustafa Suleiman discuss the implications of AI technology on the future, addressing the survival of liberal democracy and changes in employment and geopolitics.
Harari, known for his best-selling books like 'Sapiens', is joined by Suleiman, a leader in AI development and co-founder of DeepMind and Inflection AI.
Suleiman predicts that in five years, AI will be capable of planning over multiple time horizons, significantly advancing beyond generating new text.
Harari expresses concern about AI leading to the end of human-dominated history, as machines may soon make independent decisions and create new ideas.
The discussion raises ethical considerations, including the risks and potential benefits AI might bring to society, such as improved health care and innovation.
Mustafa proposes a modern Turing test involving AIâs ability to autonomously manage a $100,000 investment to create a new product, highlighting the advanced capabilities of AI.
Harari warns that the positive potential of AI, while enormous, comes with risks that may not be worth the benefits if not managed properly.
Both speakers touch on the job market transformation, suggesting that while AI may not eliminate jobs, it could drastically alter the types and distribution of jobs globally.
The conversation explores how AI could exacerbate issues of trust and communication within the political system, potentially undermining democratic processes.
Harari and Suleiman discuss the importance of regulatory frameworks to manage AI development and prevent misuse, emphasizing a balance between innovation and safety.
The role of national governments and international cooperation is questioned in the context of regulating AI, with a focus on maintaining ethical standards despite technological competition.
Suleiman highlights the collaborative efforts among AI developers to self-regulate, though recognizing that more formal regulations are necessary.
The potential for AI to be used in harmful ways, such as influencing elections or impersonating individuals, is a significant concern discussed.
Harari stresses the need for creating new institutions capable of understanding and governing AI technology effectively, ensuring they are equipped with the necessary resources and public trust.
The dialogue concludes with a contemplation of the future, where both the risks and benefits of AI are immense, requiring thoughtful and proactive governance.
Transcripts
[Music]
historian Yuval Noah Harari and
entrepreneur Mustafa Suleiman are two of
the most important voices in the
increasingly contentious debate over AI
good to be here joining us thanks for
having us
the economists got them together to
discuss what this technology means for
our future from employment and
geopolitics to the survival of liberal
democracy if the economic system has
fundamentally changed will liberal
democracy as we know it survive
[Music]
you well know Harare welcome you are a
best-selling author historian I think a
global public intellectual if not the
global public intellectual your books
from sapiens to 21 lessons from the 21st
century have sold huge numbers of copies
around the world thank you for joining
us it's good to be here Solomon
wonderful that you can join us too
you're a friend of The Economist a
fellow director on The Economist board
you are a man at The Cutting Edge of
creating the AI Revolution you are a
co-founder of deepmind you're now a
co-founder and CEO of inflection AI you
are building this future but you've also
just published a book called The Coming
wave which makes us a little concerned
about this revolution that is being
Unleashed you're both coming from
different backgrounds you are a
historian a commenter a man who I
believe doesn't use smartphones very
much not very much no
Mustafa as I know from our board
meetings is right at The Cutting Edge of
this pushing everyone to go faster so
two very different perspectives so but I
thought it would be really interesting
to bring the two of you together to have
a conversation about what is happening
what is going to happen what are the
opportunities but also what is at stake
and what are the risks so let's start
Mustafa with you
um and you are building this future so
paint us a picture of what the future is
going to be like and I'm going to give
you a time frame to keep it specific so
let's say I think you wrote in your book
that within three to five years that you
thought it was plausible that AIS could
have human level capability across a
whole range of things
so let's take five years
2028 what does the world look like how
will I interact with AIS what will we
all be doing and not doing well let's
just look back over the last 10 years to
get a sense of the trajectory that we're
on and the incredible momentum that I
think everybody can now see with the
generative AI Revolution
over the last 10 years we've become very
very good at classifying information we
can understand it we sort it label it
organize it and that classification has
been critical to enabling this next wave
because we can now read the content of
images we can understand text pretty
well we can classify audio and
transcribe it into text the machines can
now have a pretty good sense of the
conceptual representations in those
ideas the next phase of that is what
we're seeing now with the generative AI
Revolution we can now produce new images
new videos new audio and of course new
language and in the last year or so with
the rise of chat GPT and other AI models
it's pretty incredible to see how
plausible and accurate and very finesse
to these new language models are in the
next five years the Frontier Model
companies those of us at the very
Cutting Edge who are training the very
largest AI models are going to train
models that are over a thousand times
larger than what you currently see today
in GPT 4 all and with each new order of
magnitude and compute that is 10x more
compute used we tend to see really new
capabilities emerge and we predict that
the new capabilities that it will come
this time over the next five years will
be the ability to plan over multiple
time Horizons instead of just generate
new text in a one shot the model will be
able to generate a sequence of actions
over time and I think that that's really
the character of AI that we'll see in
the next five years artificial capable
AIS AIS that can't just say things they
can also do things but what does that
actually mean in practice just just use
your imagination tell me what my life
will be like in 2028 how will I interact
with them what will I do what will be
different so I've actually proposed a
modern Turing test which tries to
evaluate for exactly this point right
the last Turing test simply evaluated
for what a machine could say assuming
that what it could say represented its
intelligence now that we're kind of
approaching that moment where these AI
models are pretty good arguably they've
passed the Turing test or they maybe
they will in the next few years the real
question is how can we measure what they
can do so I've proposed a test which
involves them going off and taking a
hundred thousand dollar investment and
over the course of three months
trying to set about creating a new
product researching the market seeing
what consumers might like generating
some new images some blueprints of how
to manufacture that product contacting a
manufacturer getting it made negotiating
the price drop shipping it and then
ultimately correct collecting the
revenue and I think that over a
five-year period it's quite likely that
we will have an ACI an artificial
capable intelligence that can do the
majority of that task autonomously it
won't be able to do the whole thing
there are many tricky steps along the
way but significant portions of that it
will be able to make phone calls to
other humans to negotiate it'll be able
to call other AIS in order to establish
the right sequence in a supply chain for
example and of course it will learn to
use apis application programming
interfaces so other websites or other
knowledge bases or other information
stores and so you know the world is your
oyster you can imagine that being
applied to many many different parts of
our economy so you vote a man who
doesn't use a smartphone very much you
listen to this does this fill you with
horror or and do you agree with it do
you think that's the kind of thing that
is likely to happen in the next five
years I will take it very seriously
I don't know I'm not coming from within
the industry so I cannot comment on how
How likely it is to happen but when I
hear this as a historian for me what we
just heard this is the end of human
history not the end of History the end
of human dominated history history will
continue with somebody else in control
because what we just heard is basically
Mustafa telling us that in five years
they'll be a technology that can make
decisions independently and that can
create new ideas independently this is
the first time in history we confronted
something like this every previous
technology in history from a stone knife
to nuclear bombs it could not make
decisions like the decision to drop the
bomb on Hiroshima was not made by the
atom bomb it was made by President
Truman and similarly it can every
previous technology in history It could
only replicate our ideas like radio of
the printing press it could make copies
and disseminate the music or the poems
or the novels that some human wrote now
we have a technology that can create
completely new ideas and it can do it at
a scale far beyond what humans are
capable of so it can create new ideas
and in important areas within five years
we'll be able to enact them and that is
a profound shift before we go on to the
many ways in which this could be the end
of human history as you put it and the
the potential downsides and risks of
this can we just for a second just
indulge me I'm an optimist at heart can
we talk about the possibilities what are
the potential upsides of this because
there are many and they are really
substantial I think you you wrote that
it that there are there is the potential
that this technology can help us deal
with incredibly difficult problems and
and create tremendous honestly positive
outcomes so can we just briefly start
with that before we go down down the
road wasn't the end of human history
again I'm not I'm not talking
necessarily about the destruction of
humankind or anything like that there
are many positive potential it's just
that control Power is Shifting away from
human beings to an alien intelligence to
a non-human intelligence we'll also get
to that because there's a question of
how much power but let's stick with the
potential upsides first the
opportunities Mustafa everything that we
have created in human history is a
product of our intelligence our ability
to make predictions and then intervene
on those predictions to change the
course of the world is in a very
abstract way the way we have produced
our companies and our products and all
the value that has changed our Century I
mean if you think about it just a
century ago a kilo of grain would have
taken 50 times more labor to produce
than it does today that efficiency which
is the trajectory you have seen in
agriculture is likely to be the same
trajectory that we will see in
intelligence everything around us is a
product of intelligence and so
everything that we touch with these new
tools is likely to produce far more
value than we've ever seen before and I
think it's important to say
these are not autonomous tools by
default these these capabilities don't
just naturally emerge from the models we
attempt to engineer capabilities and the
challenge for us is to be very
deliberate and precise and careful about
those capabilities that we want to
emerge from the model that we want to
build into the model and the constraints
that we build around it it's super
important not to anthropomorphically
project ideas and you know potential
intentions or potential agency or
potential autonomy into these models the
governance challenge for us over the
next couple of decades to ensure that we
contain this wave is to ensure that we
always get to impose our constraints on
the development of this traject the the
trajectory of this development but the
capabilities that will arise will mean
for example potentially transformative
improvements in human health speeding up
the process of innovation dramatic
changes in the way scientific discovery
is done tough problems whether it's
climate change a lot of the big
challenges that we Face could be much
more easily addressed with this
capability everybody is going to have a
personal intelligence in their pocket a
smart and capable Aid a chief of staff a
research assistant constantly
prioritizing information for you putting
together the right synthesized nugget of
knowledge that you need to take action
on at any given moment and that for sure
is going to make us all much much
smarter and more capable does that part
of it sound appealing to you absolutely
I mean again if there was no positive
potential we wouldn't be sitting here
nobody would develop it nobody would
invest in it it's again it's so
appealing the positive potential is so
enormous in everything again from much
better Healthcare higher living
standards solving things like climate
change this is why it's so tempting this
is why we are willing to take the
enormous risks involved I I'm just
worried that uh in the end the deal will
not be worth it and I would comment
especially on again the notion of
intelligence
um I think it's overrated I mean Homo
sapiens at present is the most
intelligent entity on the planet it
simultaneously also the most destructive
entity on the planet and in some ways
also the most stupid entity on the
planet the only entity that that puts
the very survival of the ecosystem in
danger so you think we are trading off
more intelligence with more destructive
risk Yes again it's it's it's not uh
it's not deterministic I I don't think
that we are doomed I mean if I thought
that what's the point of talking about
it if we can't prevent the worst case
scenario well I was hoping you thought
you'd have some agency in actually
effectively we still have agency there
are a few more years I don't know how
many 5 10 30 we still have agency we are
still the ones in the driver's seat
shaping the direction this is taking no
technology is deterministic this is
something again we learned from history
you can use the same technology in
different ways you can decide which way
to develop it so we still have agency
this is why you have to think very very
carefully about what we are developing
well thinking very carefully about it is
something that Mustafa has been doing in
this book
um and I want to now go through some of
the most commonly discussed risks and I
I was trying to work out how I would go
in sort of order of Badness so I'm
starting with one that is discussed a
lot but relative to human extinction is
perhaps less bad which is the question
of jobs and will you know artificial
intelligence essentially destroy all
jobs because AIS will be better than
humans and everything you know I'm an
economist by training I you know history
suggests to me that that has never
happened before that the lump of Labor
fallacy indeed is a fallacy but tell me
what you think about that do you think
there is a risk to jobs it depends on
the time frame so over a 10 to 20 year
period my intuition and you're right
that so far the evidence doesn't support
this is that there isn't really going to
be a significant threat to jobs there's
plenty of demand there will be plenty of
work right over a 30 to 50 year time
Horizon is very difficult to speculate I
mean at the very least we can say that
two years ago we thought that these
models could never do empathy we said
that we humans were always going to
preserve kindness and understanding and
care for one another as a special skill
that humans have four years ago we said
while AIS will never be creative you
know humans will always be the creative
ones inventing new things making these
amazing leaps between new ideas is
self-evident now that both of those two
capabilities are things that these
models do incredibly well and so I think
for a period of time ai's augment our
skills they make us faster more
efficient more accurate more creative
more empathetic and so on and so forth
over a many decade period it's much
harder to say what are the set of skills
that are the permanent Preserve of the
human species given that these models
are clearly very very capable and that's
where the containment challenge really
comes in we have to make decisions we
have to decide as a species what is and
what isn't acceptable over a 30-year
period and that means politics and
governance with regard to jobs I agree
that like the the scenario that there
just won't be any jobs this is an
unlikely scenario right in in the at
least next few decades but we have to
look more carefully at time and space I
mean in terms of time the transition
period is is is the danger I mean some
jobs disappear some jobs appear people
have to transition just remember that
Hitler Rose to power in Germany because
of three years of 25 unemployment so we
are not talking about say no jobs at all
but if because of the upheavals caused
in the job market by AI we have like I
don't know three years of 25 unemployed
unemployment this could cause huge
social and political disruptions and
then the even bigger issue is one of
space that uh The Disappearance of jobs
and the new jobs will be created in
different parts of the world so we might
see a situation when there is immense
demand for more jobs in California or
Texas or China whereas entire countries
lose their uh their economic basis so
you need a lot more computer engineers
and yoga trainers and whatever in
California but you don't need any
textile workers at all in Guatemala or
Pakistan because this has all been
automated so it's not just the total
number of jobs on the planet it's the
distribution between different countries
and let's also try to remember that work
is not the goal work is not not our
desired end State we did not create
civilization so that we could have full
employment we created civilization so
that we could reduce suffering for
everybody and the Quest for abundance is
a real one we have to produce more with
less there is no way of getting rid of
the fact that population growth is set
to explode over the next Century there
are practical realities about the
demographic and Geographic and climate
trajectories that we're on which are
going to drive forward our need to
produce exactly these kinds of tools and
I think that that should be an
aspiration many many people do work that
is judging us and exhausting and tiring
and they don't find flow they don't find
their identity and it's pretty awful so
I think that we have to focus on the
prize here which is one of a question of
capturing the value that these models
will produce and then thinking about
redistribution and ultimately the
transition is exactly what's at stake we
have to manage that transition with
taxation but just with redistribution I
would say that the difficulty again the
political historical difficulty I think
there will be immense New Wealth created
by by these Technologies I'm less sure
that the governments will be able to
redistribute this wealth in a fair way
on a global level like I just don't see
the US government raising taxes on
corporations in California and sending
the money to help unemployed textile
workers in Pakistan or Guatemala kind of
retrain to for the new job market well
that actually gets us to the second
potential risk which is the risk of AI
to the political system as a whole and
you made a very um good point you are in
one of your writings where you reminded
us that liberal democracy was really
born of the Industrial Revolution and
that today's political system is really
a product of the economic system that we
are in and so there is I think a very
good fair question of if the economic
system is fundally fundamentally changed
will liberal democracy as we know it
survive yeah and on top of that it's not
just the Industrial Revolution it's the
new information Technologies of the 19th
and 20th Century before the 19th century
you don't have any example in history of
a large-scale democracy I mean you have
examples on a very small scale like in
hunter gatherer tribes or in city-states
like ancient Athens but you don't have
any example that I know of of millions
of people spread over a large territory
an entire country which managed to uh
build and maintain a democratic system
why because democracy is a conversation
and there was no information technology
in communication technology that enabled
a conversation between millions of
people over an entire country only when
first newspapers and then Telegraph and
radio and television came along this was
this became possible so modern democracy
as we know it it's built on top specific
information technology once the
information technology changes it's an
open question whether the market
obviously can survive and the biggest
danger now is the opposite than what we
face in the Middle Ages in the Middle
Ages it was impossible to have a
conversation between millions of people
because they just couldn't communicate
but in the 21st century something else
might make the conversation impossible
if trust between people collapses again
if AI if you go online which is now the
main uh way we converse on the level of
a country and the online space is
flooded by non-human entities that maybe
masquerade as human beings you talk with
someone you have no idea if it's even
human you see something you see a video
you hear an audio you have no idea if
this is really a is this true is this
fake is this a human it's not a human I
mean in this situation unless we have
some guard rails again conversation
collapses is that what you mean when you
say AI risks hacking the operating
system this is one of the things again
if if if Bots can impersonate people
it's it's basically like what happens in
in the financial system like people
invented money and it was possible to
counterfeit money to create fake money
the only way to save the financial
system from collapse was to have very
strict regulations against fake money
because the technology to create fake
money was always there so but there was
very strict regulation against it
because everybody knew if you allow fake
money to spread the financial system the
Trust In money collapses and now we are
in the analogous situation with uh the
political conversation that now it's
possible to create fake people and if we
don't ban that then trust will collapse
we'll get to the Banning or not Banning
in a minute democratizing access to the
right to broadcast has been the story of
the last 30 years hundreds of millions
of people can now create podcasts and
blogs and they're free to broadcast
their thoughts on Twitter and the
internet broadly speaking I think that
has been an incredibly positive
development you no longer have to get
access to the top newspaper or you get
the skills necessary to be part of that
institution many people at the time
feared that this would destroy our
credibility and Trust in the big news
outlets and institutions I think that
we've adapted incredibly well yes it has
been a lot of turmoil and unstable but
with every one of these new waves I
think we adjust our ability to discern
truth to dismiss nonsense and there are
both Technical and governance mechanisms
which will emerge in the next wave which
we can talk about to address things like
bot impersonation I mean I'm completely
with you I mean we should have a ban on
impersonation of digital people it
shouldn't be possible to create a
digital zany and have that be platformed
on Twitter talking all kinds of nonsense
enough with the real world
so I think that there are technical
mechanisms that we can do to prevent
those kinds of things and that's why
we're talking about them there are
mechanisms we just need to employ them I
I would say two two things first of all
it's it's a very good thing that more
people were given a voice it's diff
different with Bots Bots don't have
freedom of speech so Banning Bots
because they shouldn't have freedom of
speech they shouldn't have that's very
important yes uh there have been some
wonderful developments in the last 30
years still I'm very concerned that when
you look at countries like the United
States like the UK to some extent like
my home country of Israel I'm struck by
the fact that we have the most
sophisticated information technology in
history and we are no longer able to
talk to each other
that my impression maybe your impression
of American politics or politics in
other democracies is different my
impression is that trust is collapsing
the conversation is collapsing that
people can no longer agree who won the
last elections like the most basic fact
in a democracy who won the last it's
it's we had huge disagreements before
but I feel that now it's different that
really the conversation is breaking down
I'm not sure why but it's it's really
troubling that at the same time that we
have the really the most powerful
information technology in history and
people have no longer can talk with each
other it's a very good point we we
actually had a you may have seen it we
had a big cover package on looking at
what the impact might be in the short
term on elections and on the political
system and we concluded actually AI was
likely to have a relatively small impact
in the short term because there was
already so little trust
um so it was a sort of double-edged uh
answer you know it was it was not going
to make a huge difference but only
because things were pretty bad as they
were but you both said there needs to be
regulation
um before we get to the precisely how
the unit that we have that would do that
is the nation-state and National
governments yet you Mustafa in your book
worry that actually one of the potential
um dangers is that the powers of the
nation-state are eroded could you talk
through that as the sort of the third in
my escalating sense of risks the
challenge is that at the very moment
when we need the nation state to hold us
accountable the nation-state is
struggling under the burden of a lack of
trust and huge polarization and a
breakdown in our political process and
so combined with that the latest models
are being developed by the private
companies and by the open source it's
important to recognize it isn't just the
biggest AI developers there's a huge
proliferation of these techniques widely
available on open source code that
people can download from the web for
free and they're probably about a year
or a year and a half behind the absolute
Cutting Edge of the big models and so we
have this dual challenge like how do you
hold centralized power accountable when
the existing mechanism is basically a
little bit broken and how do you address
this Mass proliferation issue when it's
unclear how to stop anything in Mass
proliferation on the web that's a really
big challenge what we've started to see
is self-organizing initiatives on the
part of the companies right so getting
together and agreeing to sign up
proactively to self oversight both in
terms of audit in terms of capabilities
that we won't explore
etc etc now I think that's only
partially reassuring to people clearly
maybe not even reassuring at all but the
reality is I think it's the right first
step given that we haven't actually
demonstrated the large-scale harms to
arise from AIS just yet I mean this is
one of the first occasions I think in
general purpose waves of technology that
we're actually starting to adopt a
precautionary principle I'm a big
advocate of that I think that we should
be approaching a Do no harm principle
and that may mean that we have to leave
some of the benefits on the tree and
some fruit may just not be picked for a
while and we might lose some gains over
a couple of years where we may look back
in hindsight and think oh well we could
have actually gone a little bit faster
there I think that's the right trade-off
this is a moment of caution things are
accelerating extremely quickly and we
can't yet do the balance between the
harms and benefits perfectly well until
we see how this wave unfolds a little
bit so I like the fact that our company
inflection Ai and the other big
developers are trying to take a little
bit more of a cautious approach I think
that's a really interesting point
because you know we are having this
conversation you have written both of
you you extensively about the challenges
posed by this technology there's now a
parlor game amongst you know
practitioners in this world about you
know what is the risk of extinction
level events where there's a huge amount
of talk about this and I don't know in
fact I should probably ask you what
percentage of your time probably right
now it's you know close to 100 of your
time is focused on the risk since you're
promoting your book but it's it is
there's a lot of attention on this which
is which is good
um we are thinking about it early so
that gets us I think now to the most
important part of our conversation which
is what do we do and you Mustafa you lay
out a 10-point plan which is you know
the kind of action do kind of thing that
uh that someone who doesn't just comment
like you and I do but actually does
things we do so tell us what do we need
to do as as Humanity as governments as
societies to ensure that we capture the
gains from this technology but we
minimize the risks there are some very
practical things I mean so for example
red teaming these models means
adversarially testing them and trying to
put them under as much pressure as
possible to push them to generate advice
for example on how to generate a
biological or chemical weapon how to
create a bomb for example or even push
them to be very sexist racist biased in
some way and that already is pretty
significant we can see their weaknesses
I mean part of the release of these
models in the last year has given
everybody I think the opportunity to see
not just how good they are but also
their weaknesses and that is reassuring
we need to do this out in the open
that's why I'm a huge fan of the open
source Community as it is at the moment
because real developers get to play with
the models and actually see how hard it
is to produce the capabilities that
sometimes I think we fear that they're
just going to be super manipulative and
persuasive and you know destined to be
awful so that's the first thing is doing
it out in the open the second thing is
that we have to share the best practices
and so there's a competitive tension
there because safety is going to be an
asset you know I'm gonna deliver a
better product to my consumers if I have
a safer model but of course there's got
to be a requirement that if I discover a
vulnerability a weakness in the model
then I should share that just as we have
done for actually decades in many waves
of Technology not just in software
security for example but in Flight
Aviation you know the Black Box recorder
for example if there's a significant
incident not only does it record all the
Telemetry on board the aircraft but also
everything that the pilots say in the
cockpit and if there's a significant
safety incident then that's shared all
around the world with all of the
competitors which is great aircrafts are
one of the safest ways to get around
despite you know on the face of it if
you described it to an alien being 40
000 feet in the sky is a very strange
thing to do so I think there's precedent
there that we can we can follow
um
I do also agree that is probably time
for us to explicitly declare that we
should not allow these tools to be used
for electioneering I mean we cannot
trust them yet we cannot trust them to
be stable and reliable we cannot allow
people to be using them for counterfeit
digital people and clearly we've talked
about that already so there are some
capabilities which we can start to take
off the table another one would be
autonomy right right now I think
autonomy is a pretty dangerous set of
methods it's exciting it represents a
possibility that could be truly
incredible but we haven't wrapped our
hands around what the risks and
limitations are likewise training an AI
to update and improve its own code this
notion of recursive self-improvement
right closing the loop so that the AI is
in charge of defining its own goals
acquiring more resources updating its
own code with respect to some objective
these are pretty dangerous capabilities
just as we have kyc know your customer
or just as we license development
developers of nuclear technologies and
all the component involved in that
supply chain there'll be a moment where
if some of the big technology you know
providers want to experiment with those
capabilities then they should expect
there to be robust audits you know they
should expect them to be licensed and
there should be independent oversight so
how do you get that done and there seem
to be there is there are several
challenges in doing it one is the
division between the relatively few
Leading Edge models of which you have
won and then the larger tale of Open
Source models where the you know the
ability to build the model is
decentralized lots of people have access
to it my sense is that the capabilities
of the latter are a little bit behind
the capabilities of the former but they
are growing all the time and so if you
have
really considerable open source
capability what is not to stop the angry
teenager in some small town developing
capabilities that could shut down the
local hospital and how do you in your
regulatory framework guard against that
well look part of the challenge is that
these models are getting smaller and
more efficient and we know that from the
history of Technologies anything that is
useful and valuable to us gets cheaper
easier to use and it proliferates far
and wide so the destiny of this
technology over a two three four decade
period has to be proliferation and we
have to confront that reality it isn't a
contradiction to name the fact that
proliferation seems to be inevitable but
containing centralized power is an
equivalent challenge
so there is no easy answer to that I
mean Beyond surveilling the internet it
is pretty clear that in 30 years time
like you say garage tinkerers will be
able to experiment if you look at the
trajectory on synthetic biology we now
have have desk desktop synthesizers that
is the ability to engineer new synthetic
compounds they cost about twenty
thousand dollars and they basically
enable you to create potentially
molecules which are you know more
transmissible or more lethal than we had
with covid you can basically experiment
and the challenge there is that there's
no oversight you buy it off the shelf
you don't need a great deal of training
probably an undergraduate in biology
today and you'll be able to experiment
now of course they're going to get
smaller easier to use and spread far and
wide and so my book I'm really trying to
popularize the idea that this is the
defining containment challenge of the
next few decades so you use the word
containment which is interesting because
you know I'm sure the word containment
with you brings immediately you know
inspires images of George Cannon and and
you know the post-war Cold War Dynamic
and we're now you know we're in a
geopolitical world now that whether or
not you call it a new cold war is one of
great tension between the US and China
can this kind of containment as as
Mustafa calls it be done when you have
the sort of tensions you've got between
the world's big players are the you know
is the right Paradigm thinking about the
arms control treaties of the Cold War
like how do we go about doing this at a
kind of international level I think this
is the biggest problem that if it was a
question of you know humankind versus a
common threat of these new intelligent
alien agents here on Earth then yes I
think there are ways we can contain them
but if the humans are divided among
themselves
and are in an arms race then it's
because it becomes almost impossible to
contain this alien intelligence and and
there is I I'm tending to think of it
more in in terms of of really an alien
invasion
that like somebody coming and telling us
that you know there is a fleet an alien
Fleet of spaceships coming from planet
Zircon or whatever with with highly
intelligent beings they'll be here in
five years and take over the planet
maybe they'll be nice maybe they'll
solve cancer and climate change but we
are not sure
this is what we are facing except that
the aliens are not coming in spaceships
from planet Zircon that are coming from
the Laboratories
the actual characterization of the
nature of the technology an alien has by
default agency these are going to be
tools that we can apply we have narrow
settings yes but let's say they have
they potentially have agency we can try
to prevent them from having agency but
we know that they are going to be highly
intelligent and at least potentially
have agency and this is a very very
frightening mix something we never
confronted before again atom bombs
didn't have a potential for agency
printing presses did not have a
potential for agency this thing again
unless we contain it and the problem of
content is very difficult because
potentially they'll be more intelligent
than us how do you prevent something
more intelligent than you from become
from developing the agency it has I'm
not saying it's impossible I'm just
saying it it's very very difficult
I think our best bet is not to kind of
think in terms of some kind of rigid
regulation you should do this you
shouldn't do that it's in developing new
institutions
living institutions that are capable of
understanding the very fast developments
and reacting on the fly at present the
problem is that the only institutions
who really understand what is happening
are the institutions who develop the
technology the governments most of them
seem quite clueless about what's
happening also universities I mean the
amount of talent and the amount of the
the economic resources in the private
sector is far far higher than in the
universities so and again I'm I
appreciate that there are actors in the
private sector like Mustafa who are
thinking very seriously about regulation
and containment but we must have an
external entity
in in the game and for that we need to
develop new institutions that will have
the human resources that will have the
the economic and technological resources
and also will have the public trust
because without public trust it won't
work are we capable of creating such new
institutions I don't know I do think Eva
Rays is an important point which is as
we started this conversation and you
were painting the picture of five years
time and you were saying that the AIS
would be ubiquitous we'd all have our
own ones but that they would have the
capability to act not just to process
information they would have the
creativity they have now and the ability
to act but already from these generative
AI models the power that we've seen in
the last year two three four years has
been that they have been able to act in
ways that you and your other your fellow
technologists didn't anticipate they
they reached you know you didn't
anticipate you know the the speed with
which they would you would Win It Go or
so forth there was a the Striking thing
about them is that they have developed
in
unanticipatedly fast ways so if you
combine that with capability you don't
have to go as far as Yuval is saying and
saying that they're all more intelligent
than humans but there is an
unpredictability there that I think does
raise the concerns that Uval raises
which is you their creators can't quite
predict what powers they will have
they may not be fully autonomous but
they will be moving some ways towards
there and so how do you guard against
that or how do you you know red teaming
you use the phrase which is that I
understand it is that you know you keep
checking what's happening and tweak them
when you've seen what's when you
pressure test them you try to make them
fit you can't pressure test for
everything in advance so there is a I
think a very real point that Yuval is
making about as the capabilities
increase
so the risks increase of relying on you
and other Creator companies to to make I
mean it's a very fair question and
that's why I've long been calling for
the precautionary principle we should
both take some capabilities off the
table and classify those as high risk I
mean frankly the EU AI act which has
been in draft for three and a half years
is very sensible has a risk-based
framework that applies to each
application domain whether it's
Healthcare or self-driving or facial
recognition and it basically takes
certain capabilities off the table when
that threshold is exceeded I listed a
few earlier autonomy for example it's
clearly a capability that it has the
potential to be high risk recursive
self-improvement the same story so this
is the moment when we have to adopt a
precautionary principle not through any
fear-mongering but just as a logical
sensible way to proceed another model
which I think is very sensible is to
take an ipcc style approach an
international consensus around an
investigatory power to establish the
scientific fact basis for where we are
with respect to capabilities and that
has been an incredibly valuable process
set aside the negotiation and the policy
making just the evidence observing where
are we you don't have to take it from me
you should be able to take an
independent panel of experts who I would
personally Grant access to everything in
my company if they were a trusted true
impartial actor without question we
would Grant complete access and I know
that many of the other companies would
do the same again people are drawn
towards the kind of of scenario of the
AI creates a lethal virus Ebola plus
kovid and kills everybody let's go in
the more Economist Direction Financial
systems like you gave as a new touring
test the idea of AI making money what's
wrong with making money wonderful thing
so let's say that you have an AI which
has a better understanding of the
financial system than most humans most
politicians maybe most Bankers
and uh let's think back to the
2007-2008 financial crisis it started
with this I was about they called CDO
cdus this is exactly something that
these genius mathematicians invented
nobody understood them except for a
handful of Genius mathematicians in Wall
Street which is why nobody regulated
them and almost nobody saw the financial
crash coming what happens again this
kind of of apocalyptic scenario which
you don't see in Hollywood science
fiction movies the AI invents a new
class of financial devices that nobody
understands it's beyond human capability
to understand it's such complicated math
so much data nobody understands it it
makes billions of dollars billions and
billions of dollars and then it brings
down the world economy and no human
being understand what the hell is
happening like the prime ministers the
presidents the the financial ministers
what what is happening and again this is
not fantastic I mean we saw it with
human mathematicians in 2007-8 I think
that's that look that's one you know you
you can easily paint paint pictures here
that make you want to jump off the
nearest cliff and you know that's that's
one but actually my other response to
mustafa's laying out of where you say
well we just need to rule out certain
actions is to go back to the geopolitics
is it sensible for a country to rule out
certain capabilities if the other side
is not going to rule them out so you
have a you have a kind of political
economy problem going down the road that
you learn we this is a moment when we uh
collectively in the west have to
establish our values and stand behind
them what we cannot have is a race to
the bottom that says just because
they're doing it we should take the same
risk if we adopt that approach and cut
Corners left right and Center we'll
ultimately pay the price and that's not
an answer to well they're going to go
off and do it anyway we've said only
seen that with lethal autonomous weapons
I mean there's been a negotiation in the
U.N to regulate lethal autonomous
weapons for over 20 years and they
barely reached agreement on the
definition the definition of lethal
autonomous weapons let alone any
consensus so that's not great but we do
have to accept that it's the inevitable
trajectory and from our own perspective
we have to decide what we're prepared to
tolerate in society with respect to free
acting AIS facial surveillance facial
recognition and you know generally
autonomous systems I mean so far we've
taken a pretty cautious approach when we
don't have drones flying around
everywhere we can already it's totally
possible technically to autonomously fly
a drone to navigate around London we've
we've ruled it out right we don't yet
have autonomous self-driving cars even
though you know with some degree of harm
they are actually pretty well
functioning so the regulatory process is
also a cultural process of what we think
is socially and politically acceptable
at any given moment and I think an
appropriate level of caution is is what
we're seeing
much but I completely agree on that that
we need in many fields the Coalition of
the willing and if some actors in the
world don't want to join it's it's in
our interest
so again something like Banning Bots
impersonating people so some countries
will not agree but that doesn't matter
to protect our societies it's still a
very good idea to have these kinds of
regulations so that area of agreement is
one to bring us to a close but I want to
end by asking both of you and use first
Mustafa you are you know both raising
alarms but you are heavily involved in
creating this future
why do you carry on I personally believe
that it is possible to get the upsides
and minimize the downsides in the AI
that we have created Pi which stands for
personal intelligence is one of the
safest in the world today it doesn't
produce the racist toxic bias greeds
that they did two years ago it doesn't
fall victim to any of the jailbreaks The
Prompt hacks the adversarial red teams
none of those work and we've made safety
an absolute number one priority in the
design of our product so my goal has
been to do my very best to demonstrate a
path forward in the best possible way
this is an inevitable unfolding over
multiple decades this really is
happening the coming wave is coming and
I think my contribution is to try to
demonstrate in the best way that I can a
manifestation of a personal intelligence
which really does adhere to the best
safety constraints that we could
possibly think of so you've all you've
you've heard mustafa's explanation for
why he continues you look back over
human history now as you look forward is
this a technology and a pace of
innovation that Humanity will come to
regret or should Mustafa carry on it
could be gonna I can't predict the
future I would say that we invest so
much in developing artificial
intelligence and we haven't seen
anything yet like it's it's still the
very first baby steps of artificial
intelligence in terms of like you think
about I don't know the evolution of
organic life this is like the amoeba of
artificial intelligence and it won't
take millions of years to get to T-Rex
maybe it will take 20 years to get to
T-Rex and but one thing to remember is
that we also our own minds have a huge
scope for development uh also with
Humanity we haven't seen our full
potential yet and if we invest for every
dollar and minute that we invest in
artificial intelligence we invest
another dollar a minute in developing
our own Consciousness our own mind I
think we'll be okay but but I don't see
it happening I don't see this kind of
investment in in human beings that we
are seeing in in the machine well for me
this conversation with the two of you
has been just that investment thank you
both very much indeed thank you thank
you thank you
foreign
foreign
foreign
Voir Plus de Vidéos Connexes
AI, 2024 Elections & Fake Humans | Yuval Noah Harari on Morning Joe
Yuval Noah Harari - âNexusâ & Threat of AI in the Information Age | The Daily Show
Yuval Noah Harari on AI, Future Tech, Society & Global Finance
Brave New Words - Bill Gates & Sal Khan
'AI is already able to manipulate people': Expert warns of growing AI power
Mustafa Suleyman on The Coming Wave of AI, with Zanny Minton Beddoes
5.0 / 5 (0 votes)