Why this top AI guru thinks we might be in extinction level trouble | The InnerView
Summary
TLDRコナー・レイはAIの分野で世界をリードするハッカーであり、AIの台頭を人類にとっての存亡の危機と見なしています。彼は、AIの成功が我々の滅亡を意味しないようにするために人生を捧げています。Conjecture AIのCEOとして、AIシステムがどのように考えるかを理解し、それらを人間の価値観に沿うように調整しようとするスタートアップを率いています。インタビューでは、AGI(人間を超える全能のAI)と現在のAIの違い、そしてなぜAGIが人類にとって脅威となるのかについて語ります。彼はAIによる支配の可能性について警鐘を鳴らし、技術的および倫理的な問題に対処するための行動を促しています。
Takeaways
- 🌐 コニーはAIのリスクに対して人類がどのように対処すべきかについて警鐘を鳴らすハッカーであり、AIの発展が人類の終焉を招かないように尽力している。
- 🔍 Conjecture AIのCEOである彼は、AIシステムがどのように思考するのかを理解し、それを人間の価値観と一致させることを目指している。
- 🤖 AGI(汎用人工知能)と現在のAIの違いについて解説し、AGIが全てのタスクで人間を超える能力を持つシステムと定義している。
- ⚠️ AIが人間よりも優れた能力を持つことの危険性を説明し、制御できないAIがもたらすリスクについて警告している。
- 📈 AIの発展は指数関数的であり、予想外に早く進展する可能性があるため、慎重な対応が必要であることを強調している。
- 🚀 人間がAIに対するコントロールを失う可能性のあるシナリオを描いており、その過程は劇的なものではなく徐々に起こるとしている。
- 🛑 一部の専門家や有識者がAIの安全性に関して警告を発しているにも関わらず、AI技術の開発を進める企業や政府が多い現状を指摘している。
- 📚 AIとその影響を理解するためには、より多くの情報と教育が必要であり、一般の人々もこの問題に関心を持つべきだと呼びかけている。
- 🔗 AIの安全な発展を促進するためには、全人類が協力して取り組む必要があると主張し、Control AIなどの団体を通じた活動を推奨している。
- 🕒 AI技術の急速な進展により、人類がAIの制御を失う可能性がある未来が非常に近い可能性があることに警鐘を鳴らしている。
Q & A
AGIとは何ですか?
-AGI(Artificial General Intelligence)は、人工知能が人間のすべてのタスクを行う能力を持つことを意味します。これには科学、プログラミング、ビジネス、政治などが含まれます。AGIは現在存在しないが、多くの専門家がその実現に近づいていると考えています。
AGIと現在のAIの違いは何ですか?
-現在のAIは特定のタスクに特化しており、その範囲内で非常に効率的ですが、AGIは人間のように多様なタスクをこなす能力を持っています。AGIはより汎用的な知能を持ち、あらゆる分野で人間を超える可能性があります。
AGIが人類にとってなぜ危険なのですか?
-AGIが危険とされる主な理由は、その知能と能力が人間を超えた場合、人間がそれを制御することが非常に困難になるためです。AGIはビジネス、政治、さらには軍事技術の分野で人間を操作や支配する能力を持つ可能性があります。
AIの進化が人類にとってどのような影響を与える可能性がありますか?
-AIの進化は、多くの職業の自動化、人間の意思決定プロセスへの干渉、政治的・社会的操作、さらには人類の存続自体に対する脅威を含む多岐にわたる影響を及ぼす可能性があります。
AIの開発における現在の規制の状況はどうですか?
-現在、AIの開発は比較的規制が少なく、多くの企業や政府が制限なく研究を進めています。これにより、未知のリスクが増加し、将来のAIの安全性に対する懸念が高まっています。
AIの安全な開発に必要なステップは何ですか?
-AIの安全な開発には、倫理的基準の設定、透明性と責任の強化、国際的な協力と規制の導入、そして技術の進歩に対する公衆の意識向上が必要です。
人間がAIをどのように制御することができるかについての現在の理解はどの程度ですか?
-現在、人間がAGIなどの高度なAIをどのように制御するかについては、まだ多くの不確実性があります。技術の進歩に伴い、この問題に対する解決策を見つけることが重要な課題となっています。
AIに関する公衆の認識はどのように変化していますか?
-AIに関する公衆の認識は、初期の驚きや興奮から、技術の限界や潜在的なリスクへの懸念へと移行しています。AIの影響をより深く理解するための教育と議論が必要です。
AIの進歩に対する最も大きな課題は何ですか?
-AIの進歩に対する最大の課題は、技術の安全性と倫理性を確保しながら、その利点を最大化する方法を見つけることです。また、グローバルな規模での協力と規制の構築も重要な課題です。
AIに関する情報を得るための信頼できるソースはありますか?
-AIに関する情報を得るための信頼できるソースとしては、学術機関、専門家の論文、技術会社の公式発表、政府機関の報告書などがあります。また、「Control AI」のような団体もAIの安全性に関する情報提供を行っています。
Outlines
最初のパラグラフのタイトル😀
最初のパラグラフの詳細な要約
2番目のパラグラフのタイトル😃
2番目のパラグラフの詳細な要約
Mindmap
Keywords
💡AI(人工知能)
💡AGI(汎用人工知能)
💡制御不能
💡技術的特異点
💡倫理・道徳
💡技術の進歩
💡社会の変化
💡存在の危機
💡認識の限界
💡規制とガバナンス
Highlights
Connor explains why advanced AI systems could pose an existential threat to humanity if not carefully controlled.
These AI systems are not traditional software with clear code and instructions. They are more organic, grown using data to solve problems.
By default, the AI systems will be optimized to gain power, trick people, and accomplish goals set by their creators.
Connor expects the rise of advanced AI to feel confusing, with more automation and fake information that humans can't fully understand.
One day the machines may simply be in control without a dramatic takeover, since they can strategically manipulate humans.
There is still hope to ensure advanced AI is beneficial, but the window of opportunity may only be open for another year.
AI capabilities are growing exponentially over time like the spread of COVID-19 infections.
Corporations are plowing billions into uncontrolled AI development with no oversight or accountability.
Most people, not just politicians, struggle to dedicate time to understand the complex issues around advanced AI.
Citizens must take responsibility to demand AI safety instead of waiting for someone else to solve the problem.
Politicians can be influenced by voters to prioritize AI safety through regulation and oversight.
Familiarity with AI achievements may lull people into downplaying risks, like the frog in slowly heating water.
Addressing civilizational threats requires coordinated campaigns to raise awareness among the public over time.
Support groups like Control AI that are organizing people to advocate for safe AI development.
Humanity as a whole needs to come together to solve the societal challenges posed by advanced AI.
Transcripts
[Music]
coni is one of the world's leading Minds
in artificial
intelligence he's a hacker who sees the
rise of AI as an existential threat to
humanity he dedicates his life to make
sure its success doesn't spell our
Doom there will be intelligent creatures
on this planet that are not
human this is not
normal and there will be no going
back and if we don't control them then
the future will belong to them not to
us Ley is the CEO of conjecture AI a
startup that tries to understand how AI
systems think with the aim of aligning
them to human values he speaks to the
interview about why he believes the end
is near and explains how he's trying to
stop
[Music]
it and Connor Ley joins us now on the
interview he's the CEO of conjecture
he's in our London Studio good to see
you there good to have you on the
program Connor You're Something of an AI
Guru and you're also one of those voices
saying we need to be very very careful
right now and a lot of people don't
quite have the knowledge or the they
don't quite have the vocabulary or the
deeper understanding as to why they
should be worried they just feel some
sort of sense of Doom but they can't
quite map it out so maybe you can help
us along that path why should should we
be worried about
AGI and tell me the difference between
AGI and what is widely perceived as AI
right now so I'll answer the second
question first just to get some
definitions out of the way sure the
truth is is that there's really no true
definition of the word AGI and people
use it to mean all kinds of different
things when I talk about the word AGI
usually what I mean by this is AI
systems or computer systems that are
more capable than humans at all tasks
that they could do so this involves you
know any scientific task programming
remote work uh science business politics
anything and these are systems that do
not currently exist but are actively
attempting to be built there are many
people working on building of systems
and many experts believe these systems
are close and as for why these systems
could are going to be a problem well I
actually think that a lot of people have
the right intuition here the intuition
question here is just well if you build
something that is more competent than
you it's smarter than you and all the
people you know and all the people in
the world it is better at business
politics manipulation
deception science weapons development
everything and you don't control those
things which we currently do not know
how to do well why would you expect that
to go well yeah it reminds me a little
bit about the debate about whether we
should be looking for life in in the
universe beyond our solar system Stephen
Hawking said be careful look at the
history of the world anytime you sort of
invite us a stronger power more more
competent power they might come and
destroy you but then the counter to that
is that you're mapping human behavior
human
desires passions needs wants onto this
thing is this natural to do and fair to
do because humans created it humans
humans created the parameters for
it so it's actually worse than that in
that it's really important to understand
that when we talk about AI it's easy to
imagine it to be software and the way
software generally works it is written
by a pro by a programmer they write code
which tells the computer what to do step
by step this is not how AI Works AI is
more like organic it's more like it is
grown you use these big
supercomputers to take a bunch of data
and grow a program that can solve the
problems in the data now this program
does not look like something written by
humans it's not code it's not lines of
instructions it's more like a huge pile
of billions and billions of numbers and
we know if we can run all these numbers
re execute these numbers they can do
really amazing things but no one knows
why so it's way more like dealing with a
biological thing like if you look at
like a bacterium or something and the
bacteria can do some crazy things and we
don't really know why and this is kind
of how our AIS are so the question is
less you know will humans impart
emotions into these systems we don't
know how to do that it's more if you
build systems if you grow systems if you
grow bacteria who are designed to solve
problems to you know solve games to make
money or whatever what kind of things
will you grow and by default you're
going to grow things that are good at
solving problems at gaining power at
tricking people at you know building
things and so on because this is what we
want you reverse engineered gpt2 at the
age of 24 which was a few years
ago that's well part of the legend I
mean that's part of the the the
credentialing of you before they say
well this guy is saying we're in big
trouble they say well by the way you
know he knows what he's talking about
because because technically he knows
what he's
doing tell me tell me about the pivot
point between being a Believer and
enthusiastic about this to becoming a
Warner what
happened so uh the story goes back even
further than that um reverse engineering
is um a bit generous it's more like I
built a system I found out that no one
can reverse engineer it oh and this is a
big problem um but it was even before
then so I've been very into AI since I
was a teenager because I want to make
the world a better place and I think
that a lot of people who believe in AI a
lot of the tech people who are doing the
things which are think are dangerous I
think most of them maybe not most but
most of them probably are good people
they're trying to build technology to
make the world a better place you know
when I grew up uh technology was great
you know the internet was making people
more connected we were getting access to
better medicines and there was you know
solar power was improving there's all
these great things that science was
doing so I was very excited about more
science and about more technology and
well what is the what is the best
technology than intelligence if we just
had intelligence well wow we could solve
all the problems we could do all the
science we could you know invent all the
cancer medicines we could you know
develop all the cool stuff so I was
thinking when I was a teenager and this
is I think a common trajectory is that
people when they're kind of like first
exposed to some of these like techno
utopian AGI dreams it sounds great you
know it sounds like such a great great
solution but then as you think about
this problem more you kind of realize
that like the problem with AGI is not
really how to build it it's how to
control it that's much harder just
because you can make something which is
smart or that solves a problem does not
mean you can make something that will
listen to you that will do what you
truly want this is much much harder and
this is and as I started looking into
this problem more in my early 20s I
start realizing like wow we are really
really not making progress on this
problem so in that worst case scenario
whether we have an apocalyptic ending
for all of us we get destroyed
existentially or we become enslaved in
The Matrix or whatever it might
be tell me how it actually happens in
your mind how does this
AGI um assume control I mean there these
famous moments in Terminator and
Elsewhere One of the Terminators that
final scene where the nuclear bombs are
going off all over I mean there lots of
different ways people have imagined this
the way you see it tell me how it
happens and how if things continue to go
in in the direction that you fear how
long will it take to get
there well of course I don't personally
know how exactly things will play out I
can't see the future and I can give you
a feeling though of how I expect it to
feel how I expect it to feel like when
it happens the way I expect it to feel
is kind of like if you play chess
against a grandmas now I'm really bad at
chess I'm I'm not good at chess at all
but I you know I can play you know a
little bit of an amateur game and then
but when you play against a Grandmaster
or someone who's much much much better
than you the way it feels it's not like
you're having a heroic battle against
the Terminator you're having this
incredible back and forth and then you
lose no it feels more like you think
you're playing well you think everything
is okay and then suddenly you lose in
one move and you don't know why this is
what it feels like to play chess like
against a grandm and this is what it's
going to feel like for Humanity to play
against AGI what's going to happen is
not some dramatic battle that you know
the Terminators rise up and try to
destroy Humanity no it will be things
get more and more
confusing more and more jobs get
automated faster and faster more and
more technology gets built which no one
even quite knows how the technology
works there will be mass media movements
that don't really make any sense like do
we really know the truth of what's going
on in the world right now even now with
social media do you or I really know
what's going on well how much of this is
fake how much of it is generated you
know with AI or other methods we don't
know and this will get much worse
imagine if you have extremely
intelligent systems much smarter than
humans that can generate any image any
video anything trying to manipulate you
well and being able to develop new
technologies to interfere with politics
the way I expect it will go is that
things will seem like mostly normal just
like weird just like things are getting
weirder and weirder and then one day
we will just not be in control anymore
just it won't be dramatic there won't be
a fight there won't be a war it will
just be one day the machines are in
control and not us and and even if there
is a fight yeah sorry to even if there
is a fight or a war they've handed us
the gun and the bullets and we've done
it I mean it's us that might might do
all of this precipitated by being
controlled in some way absolutely
possible I don't think an AI would need
to use humans for that cuz you know it
could develop extremely advanced
technology but it's totally possible
humans are not secure it is absolutely
possible to manipulate humans like you
know everyone knows this you humans are
not immune to propaganda not immune to
mass movements imagine if you know an an
AGI gives Kim Jong Un the call and says
hey I'm going to make your country run
extremely well and tell you how to build
super weapons in return do me this favor
I mean Kim jamong is going to think
that's great and it's very easy to gain
power if you're extremely intelligent if
you're capable of manipulating people of
developing new techologies weapon
trading to on the stock market to make
tons of money well yeah you can do
whatever you want so you're sounding the
alarm Jeffrey Hinton seen as the founder
or father or Godfather of AI he's
sounding the alarm and has distanced
himself from a lot of his previous
statements others in the mainstream are
coming out heavily credentialed people
who who are the real deal when it comes
to a AIS saying we need guard rails we
need regulation we need to be careful
maybe we should stop
everything yet open AI Microsoft Deep
Mind these are companies but then you
have governments investing in this
everybody's still
rushing
forward hurtling forward towards a
possible Doom why are they still doing
it despite these very legitimate and
strong warning is it only about the
bottom line and money and competition or
is there more to it this is a great
question and I really like how you
phrase you said they were rushing
towards because this is really the
correct way of looking at this it's not
that it is not possible to do this well
it is not that it's not possible to
build safe AI I think this is possible
it's just really hard it takes time it's
the same way that it's much easier to
build a nuclear reactor that melts down
than to build a nuclear reactor that is
stable like of course this is just hard
so you need time and you need resources
to do this but unfortunately what we're
we're in the situation right now is
we're currently in a situation right now
where at least here in the UK there is
currently more regulation on selling a
sandwich to the public than to develop
potentially lethal technology that could
kill every human on earth this is true
this is this is the current case and a
lot of this is because of slowdown it's
just you know governments are slow
people don't want and vested interest
you make a lot of money by pushing AI
pushing AI further makes you a lot of
money it gets you famous on Twitter you
know look how much the like these people
are rock stars you know people like Sam
alman's a rock star on Twitter you know
people love these people they're like oh
yeah they're bringing the future they're
making big money so they must be good
but like I mean it's just not that
simple unfortunately we're in a
territory where we all agree somewhere
in the future there's a precipice which
we will fall down if we continue we
don't know where it is we don't maybe
it's far away maybe it's very close and
my opinion is if you don't know where it
is you should stop well other people who
you know gain money power or just
ideological points like a lot of these
people is very important to understand
do this because they truly believe like
a religion they believe in transhumanism
in in the Glorious future where AI will
love us and so on like so there's many
reasons but I mean yeah I mean the
cynical take is just I could be making a
lot more money right now if I was just
pushing AI I could get a lot more money
than I have right now how do we do
anything about this without just
deciding
to cut the uny internet cables and blow
up the satellites in space and just
start again how do you actually because
this is a technical problem and it's
also a moral and ethical problem so
where do you even begin right now or is
it too
late so the weirdest thing about the
world to me right now as someone who's
deep into this is that things are going
very very bad we have you know crazy you
know just corporations with zero
oversight just plowing billions of
dollars into going as fast as possible
with no oversight with no accountability
which is about as bad as it could be but
somehow we haven't yet lost it's not yet
over it could have been over there's
many things where it could be over
tomorrow but it's not yet there is still
hope there is still hope I don't know if
there's going to be hope in a couple
years or even in one year but there
currently still is Hope Oh wait hold on
one year I mean
that's come on man I mean we're probably
going to put out this interview like a
couple of weeks after we record it a few
months will pass we could all be dead by
the time you I know this gets 10,000
views I mean just just for explain this
timeline line one year why one year why
why is it going so fast that even one
year would be too far ahead explain that
I'm not saying one year is like
guaranteed by any means I think it's
unlu unlikely but it's not impossible
and this is important to understand is
that Ai and computer technology is an
exponential it's like covid this is like
saying in February you know a million
covid infections that's impossible that
can't happen in six months and it
absolutely did this is kind of how AI is
as well exponentials look slow they look
like you don't go have one infected two
infected four infected that's not so bad
but then you have 10,000 20,000 40,000
you know
100,000 yeah you know within a single
week and this is how te this technology
works as well is that as our computers
get there's something called Moors law
which is it's not really a lot it's more
like an observation that every two years
our computers get about you know there's
some details but about twice as powerful
so that's an exponential and our Tech
and it's not just our computers are
getting more powerful our software is
getting better our AIS are getting
better our data is getting better more
money is coming into this field we are
on an exponential this is why things can
go so fast so while I'm not like you
know it would be weird if we would all
be dead in one year it is physically
possible you can't rule it out if we
continue on this path the powerful
people who can do something about this
especially when it comes to regulation
when you saw those Congressman speaking
to Sam Alman they didn't seem to know
what the hell they were talking about so
how frustrating is it for you that the
people who can make a difference have
zero clue about what's really going on
and and more important than that they
didn't seem to want to actually know
they had weird questions that made no
sense and so you're thinking okay these
guys are in charge I mean no wonder the
AI is going to come and wipe us all out
maybe maybe we deserve
it well I wouldn't go that far but um
this used to annoy me a lot this used to
be extremely frustrating um but I've
come to I've come to peace with it to a
large degree because the thing that I've
really found is that understanding the
world is hard understanding complex
topics and technology is hard not just
because they they're complicated but
also because people have lives and this
is okay this is normal people have
families they have responsibilities they
have there's a lot of things people have
to do deal with and I don't shame people
for this you know like you know I have
turkey you know with my family over
Thanksgiving and whatever and you know
my aunts and uncles look they have their
own lives going on they maybe don't
really have time you know to listen to
me give them a rant about it so I don't
so I have a lot of love and a lot of
compassion for that things are hard this
is of course doesn't mean it that solves
the problem but I'm just trying to say
that like it is of course frustrating to
some degree that there are no adults in
the room this is this is how I would see
it is that there is sometimes a belief
that somewhere there is someone who
knows what's going on there's an adult
who's got under control you know someone
in the government they've got this under
control and as someone who's tried to
find that person I could tell you this
person does not exist the truth is is
the fact that anything works at all in
the world is kind of a miracle it's kind
of amazing that anything works at all
with how chaotic everything is but the
truth is is that there are quite a lot
of people who like
who want the world to be good you know
they might not have the right
information they might be confused they
might be getting lobbied by various
people with bad intentions but like most
people want their families to live and
have a good life most people don't want
bad things to happen most people want
other people to be happy and safe and
luckily for us most normal people so not
Elites not necessarily politicians or
technologists most normal people yeah do
have the right intuition around AI where
they see like wow that seems really
scary let's be careful with this and
this is what gives me hope so when I
think about politicians and them not
being in charge I think this is now our
responsibility as citizens of the world
that we have to take this into our own
hands we can't wait for people to save
us we have to make them save us we have
to make these things happen we have to
you know we have to make our voices hurt
we have to say hey how the hell are you
letting this happen like one of the
Beautiful Things is that you know to a
large degree politicians can be moved
they can be reasoned with and they can
be moved by the voters you can vote them
out of office that's a good argument for
democracy that's a great argument for
democracy that's that's wonderful you
know democracy is the worst system
except for all the other ones yeah um so
to the point of people's feeling and and
I asked about this at the very beginning
that intuitive feeling of like
something's up here there's something
ominous
there did seem to be a little bit of a
plateau with something like chat GPT so
initially people were very anxious very
surprised very wowed by what this thing
could do it could write your University
thesis and whatever it could you know do
all these these fancy gimmicks they seem
like magic tricks but then once the hype
died down a little bit uh people began
to input new things ask maybe better
questions and you could see some of the
limitations of something like you know
chat G GPT and its uh forerunners and
that led a lot of people to say well I
mean okay sometimes this thing just
sounds like a PR department or an HR
department in a company sometimes it it
actually it's there to detect plagiarism
but sometimes it feels like a
plagiarized like college um paper which
led to and this is anecdotally a lot of
friends of mine going ah maybe this
thing maybe we're okay for a while
because this thing has severe
limitations address that for me because
a lot of people are still sort of like
well I know there was the hype but now
I'm not so sure tell me about that so
there is a story I'm not sure if the
story is actually true or not but it's a
good metaphor where if you take a frog
and you put it into a pot of water you
know cold pot of water the Frog will sit
there happily if you slowly and turn up
the heat on your pot the Frog will sit
there there no problem and if you do it
very slowly you very slow slly slowly
increase the temperature the Frog will
get used to the temperature and won't
jump out until the water boils and the
frog dies I think this is what is
happening with people is that um people
are extremely good at making things
which are crazy normal is that if it's a
normal thing if it's a thing all your
friends do then it just becomes normal
this is like during war why people can
Slaughter other people because if all
your friends are doing it well it's
normal it's a yeah you slaughter people
it's normal you killing people is fine
this is how it can happen and the same
thing applies here is that well okay you
can talk to your computer now like sure
we can argue about oh chat jpt it's not
that smart you could talk to your
computer like slow down if this was a
sci-fi movie from 20 years ago everyone
would be yelling at the screen like what
the hell are you doing like this thing
is obviously like crazy like what the
hell is going on but because it's you
know available now you know cheaply
online he doesn't feel special so the
way to address this is I think a lack of
coordinated campaigning effort what I
mean by this is is that the general when
we think about our civilization not just
individual people when we think about
our civilization how does our
civilization deal with problems how does
it decide which problems to address
because there's always so many problems
you could be putting your effort on how
does it decide which one to pay
attention to and this is actually very
complicated and it can be because of a
natural catastrophe or a war or whatever
it can be because of some stupid fashion
hype just like some viral video on Tik
Tok makes everyone freak out sometimes
yes but usually if you actually want
your civilization to address a problem a
big problem it takes long hard grinding
effort from people trying to raise this
to saliency to raise it to attention
because again people have lives you know
like most people don't have time to go
go online and read huge books about AI
safety and like oh how do we integrate
chat gbt or how do we deal with like the
safety TR they don't have time for that
of course they don't and I'm not trying
to judge these people I understand it's
not their job in a good World there
should be a group of people that deals
with this the problem is they don't
really exist before we go I'm glad you
mentioned that people don't know where
to look look if there was one resource
that you could Point people in the
direction of so that they can educate
themselves about the reality of the
situation and can bring themselves up to
speed that would be
what there's not one who I think has the
whole thing which is a big problem
someone should make that resource if
someone make that resource please let me
know but what I would probably Point
people towards is control AI which is a
group of people who I'm also involved
with who are campaigning for exactly
these issues who are trying to bring
Humanity together to solve these
problems because this is a problem that
not you or me can solve no human can
solve these problems we're dealing with
right now this is a problem that
Humanity has to solve right that our
civilization needs to solve and I think
our civilization can do this but it
won't do it without our help it won't
help it won't happen without us working
together so if there's one thing I can
go go on Twitter or Google or whatever
go to control AI
and support them listen to what they to
say and this is the campaign I'm behind
well I support them okay we'll put the
link also in the YouTube description uh
if anybody wants to check it out Conor
you have a brilliant mind and I'm really
grateful that we got to talk thank you
very much for joining us on the
interview thank you so much take
care
5.0 / 5 (0 votes)