Dr. Jüergen Schmidhuber Keynote - Global AI Summit 2022

Summary

The speaker reflects on the significant advancements in AI over the past decade, highlighting breakthroughs in machine learning and deep learning that have improved healthcare, particularly in cancer detection, and enabled superhuman computer vision and self-driving cars. The evolution of neural networks, from their inception to the development of LSTM networks, is emphasized, along with the potential for AI to transform industries and enhance human life, promising a future where AI benefits all.

Takeaways

• 🏆 Victory in a machine learning competition 10 years ago marked a milestone in AI's role in healthcare, particularly in cancer detection.
• 🧠 The AI's ability to classify cells in breast tissue, a task usually requiring a trained histologist, demonstrates the power of deep learning and artificial neural networks.
• 💰 A significant decrease in computing costs has made AI more accessible and powerful, enabling advances in healthcare and other fields.
• 🚀 The progress in AI and machine learning has been rapid, with capabilities increasing exponentially over the past decades.
• 👁️ Superhuman computer vision, demonstrated in a traffic sign recognition competition, has implications for fields like self-driving cars.
• 🚗 Self-driving cars have evolved from the 1980s without GPS to today's versions enhanced by deep learning for better pattern recognition.
• 🔄 The development of LSTM (Long Short-Term Memory) networks has revolutionized sequence processing, impacting speech recognition and more.
• 📱 LSTM networks are now in billions of smartphones, enabling features like Google's speech recognition.
• 🎮 AI's ability to learn without a teacher, as demonstrated by LSTM combined with policy gradients, has led to the creation of world-class artificial video game players.
• 🌐 AI's role in healthcare continues to expand, with applications in managing diseases like diabetes and cardiovascular conditions.
• 🌟 The future of AI is bright, with the potential to transform the world significantly and improve human lives in numerous ways.

Q & A

What significant achievement in machine learning was celebrated a few days ago in the transcript?

The significant achievement celebrated was a 10-year-old victory in a machine learning competition focused on cancer detection. The AI, through deep learning and artificial neural networks, learned to classify cells in a female breast as either dangerous pre-cancer stage cells or normal cells.

How has the cost of compute changed since the AI's development 10 years ago?

The cost of compute has decreased significantly, making it 100 times cheaper than it was 10 years ago. This has allowed for greater advancements and accessibility in AI technologies.

What role does deep learning play in healthcare according to the transcript?

Deep learning plays a crucial role in healthcare by not only aiding in cancer detection but also in various other aspects. It has contributed to making human lives longer, healthier, and has been integrated into many healthcare systems.

What was the significance of the superhuman computer vision result mentioned in the transcript?

The superhuman computer vision result, achieved when compute was more than 100 times more expensive than today, was significant because it demonstrated the ability of AI to recognize traffic signs in Silicon Valley, outperforming the second-best competitor and humans. This was an important milestone for technologies such as self-driving cars.

How has the development of self-driving cars evolved since the 1980s?

Self-driving cars have come a long way since the 1980s. The first self-driving cars appeared in the late 80s without GPS or any assistance, and by 1994, they were able to navigate highway traffic at speeds of up to 180 kilometers an hour. Today's self-driving cars are more reliable, thanks to advancements in deep learning and pattern recognition.

What is the significance of the long short-term memory (LSTM) neural network?

The LSTM neural network is significant because it handles sequential data processing, which is fundamental to understanding the world through video and sound. It has been integrated into billions of smartphones for speech recognition and is widely used by companies like Google.

How has AI contributed to language translation on platforms like Facebook?

AI, specifically the LSTM neural network, has enabled the translation of one language to another with high proficiency. Facebook uses this technology to translate 30 billion messages per week, showcasing the commercial and practical applications of AI in language processing.

What is the potential future impact of AI on sustainable cities and Vision 2030?

AI's role in traffic management and healthcare is expected to be super important for sustainable cities like Neon and others. It aligns with Vision 2030 by optimizing industrial processes, logistics, and material management, contributing to the development of more efficient and livable urban environments.

How does the concept of AI learning without a teacher work?

AI can learn without a teacher through methods like policy gradients combined with LSTM. This allows AI to explore and learn complex tasks through self-discovery, setting its own goals, and conducting its own experiments, leading to continuous improvement in problem-solving abilities.

What is the historical context of neural networks mentioned in the transcript?

The historical context of neural networks spans over 200 years, starting with linear regression. Significant advancements were made in the 20th century, including the development of deep learning, backpropagation, and LSTM networks, which laid the foundation for modern AI technologies.

What is the expected future of computation and AI based on the transcript?

The future of computation and AI is expected to be revolutionary. In the near future, computational devices may match the human brain's capacity. Over the next 50 years, a device could potentially compute as much as all human brains combined, leading to transformative changes in society and technology.

Outlines

00:00

🏆 AI in Healthcare and Cancer Detection

The first paragraph discusses a significant milestone in AI, particularly in the field of healthcare and cancer detection. It talks about a machine learning competition focused on identifying cancer in cells, which was won by an AI system developed by the speaker's team. This AI, using a deep learning neural network, was the first to classify cells from a breast tissue sample to determine if they were potentially cancerous. The achievement is notable as it occurred a decade ago when computational costs were significantly higher. The speaker highlights the advancements in AI and its widespread adoption in healthcare, not just for cancer detection but for various other applications. The potential of AI to extend and improve human life is emphasized, and the hope is expressed that the data collected by organizations like Zedia will further enhance AI's role in healthcare.

05:01

🤖 Evolution of AI and Neural Networks

The second paragraph delves into the evolution of AI and neural networks, emphasizing the development and impact of long short-term memory (LSTM) networks. The speaker mentions the commercial success of LSTM in applications like speech recognition for Google and its widespread use in billions of smartphones. The paragraph also touches on the historical development of neural networks, from linear regression 200 years ago to the deep learning breakthroughs in the 1960s and 1970s. The speaker's pride in their team's contributions to the field is evident as they discuss the foundational work done in the 1990s that underpins the current popular neural networks.

10:03

🚀 Hardware Acceleration and AI's Future

The final paragraph discusses the critical role of hardware acceleration in the advancement of AI. The speaker outlines the historical trend of computational devices becoming 10 times cheaper every five years, dating back to the first programmable computer. The paragraph paints a future where AI will be even more integrated into daily life, with computational devices potentially outperforming the human brain. The speaker's company, Nations, is working to apply AI in industrial optimization, aiming to make AI accessible and beneficial for everyone. The potential for AI to revolutionize the world is compared to the advent of smartphones, with the promise of making lives longer, healthier, and happier. The speaker ends with a forward-looking perspective on the universe's future, envisioning a time when intelligence will shape and transform the cosmos.

Keywords

💡Machine Learning

Machine learning is a subset of artificial intelligence that provides systems the ability to automatically learn and improve from experience without being explicitly programmed. In the context of the video, machine learning is used in a competition for cancer detection, demonstrating its potential to enhance healthcare and extend human life by automating the classification of cells in breast tissue samples.

💡Deep Learning

Deep learning is a specialized subset of machine learning that uses artificial neural networks with many layers (hence 'deep') to model complex patterns in data. The video emphasizes the advancements in deep learning as a key factor in the progress of AI, particularly in areas like cancer detection and self-driving cars, where the ability to recognize patterns and make decisions is crucial.

💡Artificial Neural Networks

Artificial neural networks (ANNs) are computational models inspired by the human brain's neural networks. They are designed to recognize complex patterns and are capable of learning from data through a process that mirrors the way the human brain works. In the video, ANNs are highlighted as foundational to the advancements in AI, particularly in healthcare and computer vision tasks.

💡Cancer Detection

Cancer detection involves identifying the presence of cancer in the body, typically through medical tests and imaging. In the video, the application of AI in cancer detection is emphasized, particularly how machine learning and deep learning have improved the ability to classify cells and detect cancer at early stages, thereby potentially increasing survival rates.

💡Self-Driving Cars

Self-driving cars, also known as autonomous vehicles, are vehicles that use a combination of sensors, cameras, and artificial intelligence to travel between destinations without the need for human intervention. The video discusses the evolution of self-driving cars, from their early prototypes in the 1980s to the advanced AI systems of today that use deep learning for better pattern recognition and decision-making.

💡Healthcare

Healthcare refers to the provision of medical services, care, and treatment for individuals. In the video, healthcare is presented as a key area where AI, particularly through machine learning and deep learning, is making significant contributions, from cancer detection to managing other health-related data, aiming to improve human health and longevity.

💡Long Short-Term Memory (LSTM)

Long Short-Term Memory (LSTM) is a type of recurrent neural network that is particularly effective at learning long-term dependencies in data sequences. It has been instrumental in various applications such as speech recognition and language translation. In the video, LSTM is highlighted as a revolutionary neural network architecture that has been widely adopted in modern technology, including Google's speech recognition and Facebook's language translation features.

💡Sustainable Cities

Sustainable cities are urban areas designed to meet the needs of the present without compromising the ability of future generations to meet their own needs. The video discusses the role of AI, particularly in traffic management, as a key component in creating sustainable cities like Neon and others, aligning with the vision of a future where technology supports sustainable living.

💡Artificial Intelligence (AI)

Artificial Intelligence (AI) refers to the development of computer systems that can perform tasks typically requiring human intelligence, such as visual perception, speech recognition, decision-making, and language translation. In the video, AI is portrayed as a transformative technology that has the potential to significantly improve various aspects of human life, including healthcare, transportation, and communication.

💡Reinforcement Learning

Reinforcement learning is a type of machine learning where an agent learns to make decisions by taking actions in an environment to maximize some notion of cumulative reward. It has been used to train AI systems to play complex games and control robots without explicit instruction. In the video, reinforcement learning is mentioned as a method that combines with LSTM to enable AI to learn complex tasks autonomously.

💡Neural Networks

Neural networks are a series of algorithms that attempt to recognize underlying relationships in a set of data by mimicking the way the human brain operates. They are a fundamental concept in AI and have evolved over centuries to become more complex and capable. In the video, the evolution of neural networks from simple linear models to deep learning and LSTM is discussed, highlighting their foundational role in modern AI applications.

Highlights

Celebration of a 10-year-old victory in a machine learning competition focused on cancer detection, showcasing AI's potential to enhance human health.

Introduction of a deep learning artificial neural network capable of classifying cells in breast tissue, marking a significant advancement in cancer detection.

Historical achievement of superhuman computer vision in traffic sign recognition, crucial for the development of self-driving cars.

Recollection of the early development of self-driving cars in the 1980s and their progression over the decades.

The evolution of computing power and its exponential growth, enabling a millionfold increase in capabilities for AI applications.

Discussion on the broad application of deep learning techniques in traffic management and sustainable city development.

Highlighting the revolutionary impact of recurrent neural networks in processing sequential data, a fundamental aspect of human perception.

The role of Long Short-Term Memory (LSTM) networks in enabling advances in speech recognition and handwriting recognition.

Acknowledgment of LSTM's widespread adoption in smartphones and its contribution to modern digital communication.

LSTM's critical role in the development of AI capabilities for healthcare, including research on diabetes and cardiovascular diseases.

Expansion of AI applications to include creative endeavors, allowing machines to generate their own questions and conduct experiments.

A look back at the historical development of neural networks and deep learning, tracing their origins back two centuries.

Insights into the rapid advancement of computational power since the first programmable computer and its implications for future AI capabilities.

The vision of leveraging AI across various industries to optimize processes and achieve efficiency at scale.

Emphasizing the democratization of AI technology, aiming to make it accessible and beneficial for everyone.

Speculation on the future impact of AI and computational advances, suggesting a potential transformation of civilization within a few decades.

A futuristic perspective on the universe, influenced by intelligence and technology, and the limitless potential for transformation.

Transcripts

just a few days ago we celebrated the

Victory the 10 year old victory in a

machine learning competition which was

about cancer detection and our AI is

really about making human lives longer

and healthier back then this was the

first time that a deep learning

artificial neural network learned to

classify cells in a slice of a female

breast

uh as to whether they are dangerous

pre-cancer stage cells or palmness

normally you need a trained histologist

to do that but then 10 years ago when

compute was 100 times more expensive

than today we were able to achieve

through the damage

named after my brilliant postdoc

dangerous on were able to win that

competition against all kinds of other

competitors from industry and Academia

and today we can one do 100 times as

much for the same price and everybody is

using that now in healthcare not only

for cancer detection but many other

things and we hope that the data of

Sudan

uh which is collecting lots of

healthcare relevant data is going to

help

um to achieve within the with with

the help of deep learning and artificial

neural networks

um

there I'm getting a little time is going

to help to really prolong lives in the

kingdom and Beyond

now uh even one year before that for the

first time we had a

in a competition a superhuman computer

vision resulted when compute was more

than 100 times more expensive than today

again through the damn net and that was

about traffic sign recognition in

Silicon Valley there was a competition

and um and we were three times better

than the second best

guy and twice as good as humans even

back then and of course all of that is

important for self-driving cars

self-driving cars are an old thing the

first self-driving cars appeared in the

1980s in my hometown in Munich

in the late 80s Anne's Dickman's and his

team the first self-driving cars without

GPS and any help like that and by 1994

that is

almost three decades ago these

self-driving cars were in highway

traffic

going up to 180 kilometers an hour

passing other cars almost three decades

ago today computers are a million times

faster for the same price and we can do

a million times as much and one has to

admit though that today's self-driving

cars are more reliable because they use

deep learning and techniques like I just

mentioned to become much better pattern

recognizers they aren't perfect yet but

getting better and better

the techniques that we have used for the

these competitions are now also widely

used for traffic management and all of

that is going to be super important for

sustainable cities like neon and other

cities in the kingdom and Beyond in line

with vision 2030

no

um even before that in the year 2009

our our

deep free current neural network for the

first time I was able to win

competitions in in all kinds of

applications where

um where it's about processing sequences

most of the world is sequential all of

the world is sequential and what you

really are exposed to during your life

is video coming in through your cameras

and and sound coming in through your

microphones and how to process that with

a standard neural network you cannot do

it now you need recurrent connections

and um and the standard for recurrent

connections is this long short-term

memory that we first published in a

journal in 1997 uh but then 2006

um through a method called

connectionless temporary classification

it was able to

um to

outperform all existing methods in

topics such as handwriting connective

handwriting recognition and I think I

hope that the first author of this paper

is in the audience it's Alex Graves the

CTO of Nations our company

and I think our CEO of Faustino Gomez is

also somewhere in the audience he's a

co-author of this paper and then um this

long short-term memory really took off

and today you have it in your pockets on

your smartphone it's on billions of

smartphones now and it's doing the

speech recognition for Google and for

many other companies and I cannot

explain it in detail but at least I can

mention the names of the brilliant

students in my lab who made it possible

first of all right already in the early

90s and then Felix gears and Alex Graves

and others who made that possible now

Bloomberg called the most commercial AI

achievement why because not only because

it's now the most cited neural network

of the 20th century no because it's

really

permitting some Modern World on billions

of machines for example if you are on

Facebook there's a little button and you

can translate from one language to the

next already in 2017 lstm this learning

artificial neural network long

short-term memory is the name was able

to

um

to learn to translate from one language

to another and back then Facebook used

that 30 billion times per week to

translate 30 billion messages per week

so um

a lot of that is

already today defining

um the modern world in many ways and

it's also again used widely for

healthcare if you Google lstm and

Healthcare topics such as diabetes or

cardiovascular disease you will find

lots of papers just on combinations of

these things and um again we see that

our AI is really

helping to improve Healthcare and make

human lives longer and healthier which

is hopefully going to be supported

further in the future so the data

gathered by zedia who is organizing this

big conference here

now since lstm can also be trained

without any teacher to do stuff

um such as controlling robots or playing

video games and um although we published

that in 2007 for the first time

um a combination of lstm and a method

called policy gradients to learn without

a teacher complex things

very recently in a deep mind a famous

company and open AI another famous

company they used to build the best the

world's best artificial video game

players you have to know video game

playing is harder than chess chess and

go are easier because the current input

of the board tells you everything that

you need to know about an optimal next

move and this is not the case in video

games where you have to have a memory of

past events so

there you need recurrent artificial

neural networks like long short-term

memory uh what's very important to me is

to point out that we are not only having

neural networks that learn by imitating

human teachers or by following

goals defined by human supervisors no we

also have

creative

careers artificial AI is rich

ask themselves their own questions and

set themselves their own goals and

invent their own experiments to

figure out how does the world work and

what can you do in it and in the course

of these self-invented experiments they

become better and better

um a problem solvers so it's it's not

true that Curiosity and creativity are

limited to humans no we have that in

machines and we have had that actually

for decades in machines all of that is

of course building on lots of

insights of past centuries the first

neural networks the first linear neural

networks appear 200 years ago they

weren't called linear neural networks

they were called linear regression but

mathematically it's exactly the same

thing and then in 1965 for the first

time we had not only shallow learning

but deep learning in the Ukraine this

was invented in the Ukraine by iwaknenko

and lapa and then soon networks became

bigger and deeper and and in 1970 a

famous method called back publication

was invented in Finland by sebolina inma

now widely used to train neural networks

and then in 1990 we had our miraculous

year where lots of the things that you

find now in your smartphone have their

roots and I won't have time to go into

the details of that but proudly I can

claim today at least that the five most

popular neural networks all are based on

stuff that we did in the 90s back then

so uh without the enormous Hardware

acceleration of recent decades all of

that would have been in vain

but luckily every five years computers

getting 10 times cheaper and this is an

old Trend that has held since 1941 when

Conrad Souza built the first program

controlled computer in Berlin and he

could do roughly one operation per

second but now after many decades of

getting a fact of 100 per decade we can

do almost a billion billion operations

not quite but almost at the same price

and very soon we are going to

have little computational devices that

not only can compute as much as a human

brain not we don't have that yet but

honest we are there but then 50 years

later if the trend doesn't break a

little computational device will compute

as much as all human brains combined and

you can imagine that um that everything

is going to change then

our company Nations is trying to

leverage all of that by applying it to

industrially I

providing some unique products for

industrial companies where there are

lots of industrial processes that you

want to optimize from Logistics to

material

[Music]

management processes and whatever and I

think this is going to be a huge thing

in the future my almost final slide is

going to be first one that's our logo

which is really about AI for all AI is

not going to be uh

just making profits for a couple of huge

companies no it's going to be for

everyone everyone is going to have more

and more and faster and faster and

better and better AI is working for him

making their lives

um easier and um

and more livable in many many ways so

just like smartphones are much cheaper

today than 40 years ago when just a few

people had these mobile phones and

they're Porsches

we will have a revolution in the sense

that everybody is going to profit

greatly from Ai and and it's really

about making people people's lives

longer and healthier and and easier

and happier

and thou now I don't have any time for

this final slide which is about the far

future and the far past but those of you

who have a camera maybe you want to take

a picture and uh ponder it at home uh

it's basically about the historic

context of all of that

and when I say historic context I mean

the time since the Big Bang 13.8 billion

years ago and let me quickly give you

the fast version the short version

Big Bang

13.8 billion years ago we divide by 1

000 we come out 13.5 million years ago

when um the first hominids emerged

something very important that happened

back then and everything that we

consider important happened in these

past 13.5 million years now we take

these 13.5 million years and divide

Again by a factor of 1000 and everything

we consider important happened in these

past 13 000 years when civilization was

invented civilization around 13 000

years ago agriculture domestication are

the animals and so on and now in the

very near future for the first time we

are going to have little machines that

can compute as much as a human brain and

it's not going to stop there and a new

huge Revolution is coming and maybe then

we will have to divide Again by a factor

of 1000 and within 30 13 years again the

entire world is going to change as much

as it changed in the past 13 000 years

or the past 13.5 million years

so huge things are coming and um and um

the far future you can easily deduce if

you don't divide by a fact of 1000 but

multiply by a fact of one thousand now

look ahead to a time when the universe

is going to be a thousand times older

than it is now it is still very young

the universe it's going to be much older

than it is now and it's going to be

totally within the limits of light speed

and physics it's going to be completely

colonized and transformed and um and

shaped by intelligence in a way that we

cannot imagine but I think it's

something

um to look forward to

Don't Be Afraid all will be good thank

you for your attention

[Applause]