Dr. Jüergen Schmidhuber Keynote - Global AI Summit 2022
Summary
TLDRThe speaker reflects on the significant advancements in AI over the past decade, highlighting breakthroughs in machine learning and deep learning that have improved healthcare, particularly in cancer detection, and enabled superhuman computer vision and self-driving cars. The evolution of neural networks, from their inception to the development of LSTM networks, is emphasized, along with the potential for AI to transform industries and enhance human life, promising a future where AI benefits all.
Takeaways
- 🏆 Victory in a machine learning competition 10 years ago marked a milestone in AI's role in healthcare, particularly in cancer detection.
- 🧠 The AI's ability to classify cells in breast tissue, a task usually requiring a trained histologist, demonstrates the power of deep learning and artificial neural networks.
- 💰 A significant decrease in computing costs has made AI more accessible and powerful, enabling advances in healthcare and other fields.
- 🚀 The progress in AI and machine learning has been rapid, with capabilities increasing exponentially over the past decades.
- 👁️ Superhuman computer vision, demonstrated in a traffic sign recognition competition, has implications for fields like self-driving cars.
- 🚗 Self-driving cars have evolved from the 1980s without GPS to today's versions enhanced by deep learning for better pattern recognition.
- 🔄 The development of LSTM (Long Short-Term Memory) networks has revolutionized sequence processing, impacting speech recognition and more.
- 📱 LSTM networks are now in billions of smartphones, enabling features like Google's speech recognition.
- 🎮 AI's ability to learn without a teacher, as demonstrated by LSTM combined with policy gradients, has led to the creation of world-class artificial video game players.
- 🌐 AI's role in healthcare continues to expand, with applications in managing diseases like diabetes and cardiovascular conditions.
- 🌟 The future of AI is bright, with the potential to transform the world significantly and improve human lives in numerous ways.
Q & A
What significant achievement in machine learning was celebrated a few days ago in the transcript?
-The significant achievement celebrated was a 10-year-old victory in a machine learning competition focused on cancer detection. The AI, through deep learning and artificial neural networks, learned to classify cells in a female breast as either dangerous pre-cancer stage cells or normal cells.
How has the cost of compute changed since the AI's development 10 years ago?
-The cost of compute has decreased significantly, making it 100 times cheaper than it was 10 years ago. This has allowed for greater advancements and accessibility in AI technologies.
What role does deep learning play in healthcare according to the transcript?
-Deep learning plays a crucial role in healthcare by not only aiding in cancer detection but also in various other aspects. It has contributed to making human lives longer, healthier, and has been integrated into many healthcare systems.
What was the significance of the superhuman computer vision result mentioned in the transcript?
-The superhuman computer vision result, achieved when compute was more than 100 times more expensive than today, was significant because it demonstrated the ability of AI to recognize traffic signs in Silicon Valley, outperforming the second-best competitor and humans. This was an important milestone for technologies such as self-driving cars.
How has the development of self-driving cars evolved since the 1980s?
-Self-driving cars have come a long way since the 1980s. The first self-driving cars appeared in the late 80s without GPS or any assistance, and by 1994, they were able to navigate highway traffic at speeds of up to 180 kilometers an hour. Today's self-driving cars are more reliable, thanks to advancements in deep learning and pattern recognition.
What is the significance of the long short-term memory (LSTM) neural network?
-The LSTM neural network is significant because it handles sequential data processing, which is fundamental to understanding the world through video and sound. It has been integrated into billions of smartphones for speech recognition and is widely used by companies like Google.
How has AI contributed to language translation on platforms like Facebook?
-AI, specifically the LSTM neural network, has enabled the translation of one language to another with high proficiency. Facebook uses this technology to translate 30 billion messages per week, showcasing the commercial and practical applications of AI in language processing.
What is the potential future impact of AI on sustainable cities and Vision 2030?
-AI's role in traffic management and healthcare is expected to be super important for sustainable cities like Neon and others. It aligns with Vision 2030 by optimizing industrial processes, logistics, and material management, contributing to the development of more efficient and livable urban environments.
How does the concept of AI learning without a teacher work?
-AI can learn without a teacher through methods like policy gradients combined with LSTM. This allows AI to explore and learn complex tasks through self-discovery, setting its own goals, and conducting its own experiments, leading to continuous improvement in problem-solving abilities.
What is the historical context of neural networks mentioned in the transcript?
-The historical context of neural networks spans over 200 years, starting with linear regression. Significant advancements were made in the 20th century, including the development of deep learning, backpropagation, and LSTM networks, which laid the foundation for modern AI technologies.
What is the expected future of computation and AI based on the transcript?
-The future of computation and AI is expected to be revolutionary. In the near future, computational devices may match the human brain's capacity. Over the next 50 years, a device could potentially compute as much as all human brains combined, leading to transformative changes in society and technology.
Outlines
🏆 AI in Healthcare and Cancer Detection
The first paragraph discusses a significant milestone in AI, particularly in the field of healthcare and cancer detection. It talks about a machine learning competition focused on identifying cancer in cells, which was won by an AI system developed by the speaker's team. This AI, using a deep learning neural network, was the first to classify cells from a breast tissue sample to determine if they were potentially cancerous. The achievement is notable as it occurred a decade ago when computational costs were significantly higher. The speaker highlights the advancements in AI and its widespread adoption in healthcare, not just for cancer detection but for various other applications. The potential of AI to extend and improve human life is emphasized, and the hope is expressed that the data collected by organizations like Zedia will further enhance AI's role in healthcare.
🤖 Evolution of AI and Neural Networks
The second paragraph delves into the evolution of AI and neural networks, emphasizing the development and impact of long short-term memory (LSTM) networks. The speaker mentions the commercial success of LSTM in applications like speech recognition for Google and its widespread use in billions of smartphones. The paragraph also touches on the historical development of neural networks, from linear regression 200 years ago to the deep learning breakthroughs in the 1960s and 1970s. The speaker's pride in their team's contributions to the field is evident as they discuss the foundational work done in the 1990s that underpins the current popular neural networks.
🚀 Hardware Acceleration and AI's Future
The final paragraph discusses the critical role of hardware acceleration in the advancement of AI. The speaker outlines the historical trend of computational devices becoming 10 times cheaper every five years, dating back to the first programmable computer. The paragraph paints a future where AI will be even more integrated into daily life, with computational devices potentially outperforming the human brain. The speaker's company, Nations, is working to apply AI in industrial optimization, aiming to make AI accessible and beneficial for everyone. The potential for AI to revolutionize the world is compared to the advent of smartphones, with the promise of making lives longer, healthier, and happier. The speaker ends with a forward-looking perspective on the universe's future, envisioning a time when intelligence will shape and transform the cosmos.
Mindmap
Keywords
💡Machine Learning
💡Deep Learning
💡Artificial Neural Networks
💡Cancer Detection
💡Self-Driving Cars
💡Healthcare
💡Long Short-Term Memory (LSTM)
💡Sustainable Cities
💡Artificial Intelligence (AI)
💡Reinforcement Learning
💡Neural Networks
Highlights
Celebration of a 10-year-old victory in a machine learning competition focused on cancer detection, showcasing AI's potential to enhance human health.
Introduction of a deep learning artificial neural network capable of classifying cells in breast tissue, marking a significant advancement in cancer detection.
Historical achievement of superhuman computer vision in traffic sign recognition, crucial for the development of self-driving cars.
Recollection of the early development of self-driving cars in the 1980s and their progression over the decades.
The evolution of computing power and its exponential growth, enabling a millionfold increase in capabilities for AI applications.
Discussion on the broad application of deep learning techniques in traffic management and sustainable city development.
Highlighting the revolutionary impact of recurrent neural networks in processing sequential data, a fundamental aspect of human perception.
The role of Long Short-Term Memory (LSTM) networks in enabling advances in speech recognition and handwriting recognition.
Acknowledgment of LSTM's widespread adoption in smartphones and its contribution to modern digital communication.
LSTM's critical role in the development of AI capabilities for healthcare, including research on diabetes and cardiovascular diseases.
Expansion of AI applications to include creative endeavors, allowing machines to generate their own questions and conduct experiments.
A look back at the historical development of neural networks and deep learning, tracing their origins back two centuries.
Insights into the rapid advancement of computational power since the first programmable computer and its implications for future AI capabilities.
The vision of leveraging AI across various industries to optimize processes and achieve efficiency at scale.
Emphasizing the democratization of AI technology, aiming to make it accessible and beneficial for everyone.
Speculation on the future impact of AI and computational advances, suggesting a potential transformation of civilization within a few decades.
A futuristic perspective on the universe, influenced by intelligence and technology, and the limitless potential for transformation.
Transcripts
just a few days ago we celebrated the
Victory the 10 year old victory in a
machine learning competition which was
about cancer detection and our AI is
really about making human lives longer
and healthier back then this was the
first time that a deep learning
artificial neural network learned to
classify cells in a slice of a female
breast
uh as to whether they are dangerous
pre-cancer stage cells or palmness
normally you need a trained histologist
to do that but then 10 years ago when
compute was 100 times more expensive
than today we were able to achieve
through the damage
named after my brilliant postdoc
dangerous on were able to win that
competition against all kinds of other
competitors from industry and Academia
and today we can one do 100 times as
much for the same price and everybody is
using that now in healthcare not only
for cancer detection but many other
things and we hope that the data of
Sudan
uh which is collecting lots of
healthcare relevant data is going to
help
um to achieve within the with with
the help of deep learning and artificial
neural networks
um
there I'm getting a little time is going
to help to really prolong lives in the
kingdom and Beyond
now uh even one year before that for the
first time we had a
in a competition a superhuman computer
vision resulted when compute was more
than 100 times more expensive than today
again through the damn net and that was
about traffic sign recognition in
Silicon Valley there was a competition
and um and we were three times better
than the second best
guy and twice as good as humans even
back then and of course all of that is
important for self-driving cars
self-driving cars are an old thing the
first self-driving cars appeared in the
1980s in my hometown in Munich
in the late 80s Anne's Dickman's and his
team the first self-driving cars without
GPS and any help like that and by 1994
that is
almost three decades ago these
self-driving cars were in highway
traffic
going up to 180 kilometers an hour
passing other cars almost three decades
ago today computers are a million times
faster for the same price and we can do
a million times as much and one has to
admit though that today's self-driving
cars are more reliable because they use
deep learning and techniques like I just
mentioned to become much better pattern
recognizers they aren't perfect yet but
getting better and better
the techniques that we have used for the
these competitions are now also widely
used for traffic management and all of
that is going to be super important for
sustainable cities like neon and other
cities in the kingdom and Beyond in line
with vision 2030
no
um even before that in the year 2009
our our
deep free current neural network for the
first time I was able to win
competitions in in all kinds of
applications where
um where it's about processing sequences
most of the world is sequential all of
the world is sequential and what you
really are exposed to during your life
is video coming in through your cameras
and and sound coming in through your
microphones and how to process that with
a standard neural network you cannot do
it now you need recurrent connections
and um and the standard for recurrent
connections is this long short-term
memory that we first published in a
journal in 1997 uh but then 2006
um through a method called
connectionless temporary classification
it was able to
um to
outperform all existing methods in
topics such as handwriting connective
handwriting recognition and I think I
hope that the first author of this paper
is in the audience it's Alex Graves the
CTO of Nations our company
and I think our CEO of Faustino Gomez is
also somewhere in the audience he's a
co-author of this paper and then um this
long short-term memory really took off
and today you have it in your pockets on
your smartphone it's on billions of
smartphones now and it's doing the
speech recognition for Google and for
many other companies and I cannot
explain it in detail but at least I can
mention the names of the brilliant
students in my lab who made it possible
first of all right already in the early
90s and then Felix gears and Alex Graves
and others who made that possible now
Bloomberg called the most commercial AI
achievement why because not only because
it's now the most cited neural network
of the 20th century no because it's
really
permitting some Modern World on billions
of machines for example if you are on
Facebook there's a little button and you
can translate from one language to the
next already in 2017 lstm this learning
artificial neural network long
short-term memory is the name was able
to
um
to learn to translate from one language
to another and back then Facebook used
that 30 billion times per week to
translate 30 billion messages per week
so um
a lot of that is
already today defining
um the modern world in many ways and
it's also again used widely for
healthcare if you Google lstm and
Healthcare topics such as diabetes or
cardiovascular disease you will find
lots of papers just on combinations of
these things and um again we see that
our AI is really
helping to improve Healthcare and make
human lives longer and healthier which
is hopefully going to be supported
further in the future so the data
gathered by zedia who is organizing this
big conference here
now since lstm can also be trained
without any teacher to do stuff
um such as controlling robots or playing
video games and um although we published
that in 2007 for the first time
um a combination of lstm and a method
called policy gradients to learn without
a teacher complex things
very recently in a deep mind a famous
company and open AI another famous
company they used to build the best the
world's best artificial video game
players you have to know video game
playing is harder than chess chess and
go are easier because the current input
of the board tells you everything that
you need to know about an optimal next
move and this is not the case in video
games where you have to have a memory of
past events so
there you need recurrent artificial
neural networks like long short-term
memory uh what's very important to me is
to point out that we are not only having
neural networks that learn by imitating
human teachers or by following
goals defined by human supervisors no we
also have
creative
careers artificial AI is rich
ask themselves their own questions and
set themselves their own goals and
invent their own experiments to
figure out how does the world work and
what can you do in it and in the course
of these self-invented experiments they
become better and better
um a problem solvers so it's it's not
true that Curiosity and creativity are
limited to humans no we have that in
machines and we have had that actually
for decades in machines all of that is
of course building on lots of
insights of past centuries the first
neural networks the first linear neural
networks appear 200 years ago they
weren't called linear neural networks
they were called linear regression but
mathematically it's exactly the same
thing and then in 1965 for the first
time we had not only shallow learning
but deep learning in the Ukraine this
was invented in the Ukraine by iwaknenko
and lapa and then soon networks became
bigger and deeper and and in 1970 a
famous method called back publication
was invented in Finland by sebolina inma
now widely used to train neural networks
and then in 1990 we had our miraculous
year where lots of the things that you
find now in your smartphone have their
roots and I won't have time to go into
the details of that but proudly I can
claim today at least that the five most
popular neural networks all are based on
stuff that we did in the 90s back then
so uh without the enormous Hardware
acceleration of recent decades all of
that would have been in vain
but luckily every five years computers
getting 10 times cheaper and this is an
old Trend that has held since 1941 when
Conrad Souza built the first program
controlled computer in Berlin and he
could do roughly one operation per
second but now after many decades of
getting a fact of 100 per decade we can
do almost a billion billion operations
not quite but almost at the same price
and very soon we are going to
have little computational devices that
not only can compute as much as a human
brain not we don't have that yet but
honest we are there but then 50 years
later if the trend doesn't break a
little computational device will compute
as much as all human brains combined and
you can imagine that um that everything
is going to change then
our company Nations is trying to
leverage all of that by applying it to
industrially I
providing some unique products for
industrial companies where there are
lots of industrial processes that you
want to optimize from Logistics to
material
[Music]
management processes and whatever and I
think this is going to be a huge thing
in the future my almost final slide is
going to be first one that's our logo
which is really about AI for all AI is
not going to be uh
just making profits for a couple of huge
companies no it's going to be for
everyone everyone is going to have more
and more and faster and faster and
better and better AI is working for him
making their lives
um easier and um
and more livable in many many ways so
just like smartphones are much cheaper
today than 40 years ago when just a few
people had these mobile phones and
they're Porsches
we will have a revolution in the sense
that everybody is going to profit
greatly from Ai and and it's really
about making people people's lives
longer and healthier and and easier
and happier
and thou now I don't have any time for
this final slide which is about the far
future and the far past but those of you
who have a camera maybe you want to take
a picture and uh ponder it at home uh
it's basically about the historic
context of all of that
and when I say historic context I mean
the time since the Big Bang 13.8 billion
years ago and let me quickly give you
the fast version the short version
Big Bang
13.8 billion years ago we divide by 1
000 we come out 13.5 million years ago
when um the first hominids emerged
something very important that happened
back then and everything that we
consider important happened in these
past 13.5 million years now we take
these 13.5 million years and divide
Again by a factor of 1000 and everything
we consider important happened in these
past 13 000 years when civilization was
invented civilization around 13 000
years ago agriculture domestication are
the animals and so on and now in the
very near future for the first time we
are going to have little machines that
can compute as much as a human brain and
it's not going to stop there and a new
huge Revolution is coming and maybe then
we will have to divide Again by a factor
of 1000 and within 30 13 years again the
entire world is going to change as much
as it changed in the past 13 000 years
or the past 13.5 million years
so huge things are coming and um and um
the far future you can easily deduce if
you don't divide by a fact of 1000 but
multiply by a fact of one thousand now
look ahead to a time when the universe
is going to be a thousand times older
than it is now it is still very young
the universe it's going to be much older
than it is now and it's going to be
totally within the limits of light speed
and physics it's going to be completely
colonized and transformed and um and
shaped by intelligence in a way that we
cannot imagine but I think it's
something
um to look forward to
Don't Be Afraid all will be good thank
you for your attention
[Applause]
浏览更多相关视频
Understanding Artificial Intelligence and Its Future | Neil Nie | TEDxDeerfield
Welcome (Deep Learning Specialization C1W1L01)
Inteligência artificial: o que é, história e definição
How Powerful Will AI Be In 5 Years?
Introduction
The Rise of Artificial Intelligence through Deep Learning | Yoshua Bengio | TEDxMontreal
5.0 / 5 (0 votes)