The Rise of Artificial Intelligence through Deep Learning | Yoshua Bengio | TEDxMontreal

Summary

This video explores the impact of artificial intelligence (AI) on society, highlighting how machine learning is transforming our cognitive abilities. The speaker discusses AI breakthroughs, such as AlphaGo's mastery of the game of Go, and the role of deep learning in speech recognition, image processing, and language translation. The importance of unsupervised learning and its future potential, including applications in self-driving cars and personalized medicine, is emphasized. The speaker also stresses the need for public involvement in shaping AI's future, advocating for its use in socially beneficial areas while addressing ethical concerns.

Takeaways

• 🤖 Artificial Intelligence (AI) is bringing a new industrial revolution, expanding our cognitive abilities.
• 🎮 AlphaGo, an AI system, beat the world champion in the game of Go using deep learning, marking a major milestone in AI.
• 🧠 Machine learning allows computers to learn from data, making it a key approach in developing intelligence in machines.
• 🧩 Humans have intuitive knowledge that's hard to program into computers, but machine learning enables computers to learn on their own.
• 🔑 Neural networks and deep learning, inspired by the brain, have revolutionized AI by achieving breakthroughs in tasks like speech recognition, image recognition, and language translation.
• 👶 Even a two-year-old child has intuitive knowledge about physics, which AI systems still struggle to learn – highlighting the gap in unsupervised learning.
• 🚗 Unsupervised learning could help AI reason and plan ahead, making technologies like self-driving cars more reliable and adaptable.
• 🌌 Deep learning models can generate new and plausible images from learned data, but they are still imperfect and missing key details that humans would recognize.
• 🏫 Montreal has become a hub for AI research, with significant investments and a growing number of students and startups in the field.
• 🌍 AI has immense potential for social good, like improving healthcare, legal services, and personalized medicine. Public involvement is crucial to guide AI's future societal impact.

Q & A

What distinguishes the current industrial revolution driven by AI from previous industrial revolutions?

Previous industrial revolutions expanded mechanical power, while the current AI-driven revolution, known as the second machine age, expands cognitive abilities, allowing computers to replace not just manual labor, but also mental labor.

What key event in AI development is highlighted in the script, and why is it significant?

The script highlights the success of AlphaGo in beating the world champion at the game of Go, which is significant because Go is more complex than chess for computers to master, marking a major milestone in AI's cognitive capabilities through deep learning.

How does machine learning differ from traditional approaches to AI?

Traditional AI struggled with programming intuitive knowledge into computers, while machine learning allows computers to learn from data and examples autonomously, enabling them to acquire knowledge that is difficult to explicitly program.

What are neural networks, and how have they contributed to recent AI breakthroughs?

Neural networks, inspired by the structure of the human brain, have significantly advanced AI, enabling breakthroughs in areas like speech recognition, computer vision, and language translation by allowing computers to learn through layers of data representation.

Why is unsupervised learning considered a key challenge in AI research?

Unsupervised learning is challenging because it involves discovering patterns in data without explicit guidance. Unlike humans, who can learn concepts like intuitive physics naturally, AI still struggles to autonomously learn such unsupervised knowledge.

How do neural networks mimic the human brain in processing visual data?

Neural networks process visual data similarly to the human brain, starting with basic recognition of edges and shapes in early layers (like the visual cortex in humans) and progressing to more abstract, high-level representations such as object recognition.

What is supervised learning, and how is it currently used in AI training?

Supervised learning involves training AI by providing it with labeled data, where humans manually tell the computer what each image or piece of data represents. While effective, it is labor-intensive, as machines require millions of examples to learn.

Why is unsupervised learning seen as important for the future of self-driving cars?

Unsupervised learning allows AI to predict and simulate plausible future scenarios based on current conditions, enabling self-driving cars to anticipate and plan for unfamiliar situations without requiring explicit training for every possible event.

What are some social implications of AI mentioned in the script?

The script highlights the potential for AI to provide personalized medicine, improve access to legal services, and address other social issues by democratizing access to services that are currently out of reach for many people due to cost or availability.

What role does the speaker suggest ordinary people should play in the future of AI?

The speaker encourages ordinary people to learn about AI and its underlying principles, so they can participate in important decisions about its future development, ensuring AI is used for the collective good and not just for commercial or military purposes.

Outlines

00:00

🤖 The Rise of AI: Expanding Human Cognitive Power

The speaker introduces the transformative impact of AI, highlighting its role in ushering in a new industrial revolution. Unlike previous revolutions that enhanced mechanical power, AI focuses on augmenting cognitive abilities. The example of AlphaGo, which defeated a human champion at the game of Go, is cited to demonstrate how machine learning enables computers to learn from data. This revolution is compared to human intuition, emphasizing that AI will eventually replicate intuitive knowledge, though challenges still remain in fully programming computers with such abilities.

05:02

🌐 Deep Learning's Role in Language Translation

The speaker discusses how deep learning is being utilized for language translation, with examples like Google Translate. AI's progress in understanding natural language is emphasized, though the speaker cautions that AI is still far from human-level understanding. A comparison is made between a two-year-old child’s grasp of intuitive physics and AI’s limitations, illustrating the gap between human and machine learning. Unsupervised learning, which is crucial for AI to self-discover knowledge, is introduced as one of the next major hurdles for the AI community to overcome.

10:03

🚗 Unsupervised Learning and the Future of Self-Driving Cars

The speaker elaborates on how unsupervised learning can help AI project future scenarios, a crucial skill for applications like self-driving cars. Unlike supervised learning, where machines need explicit training data for every situation, unsupervised learning allows for flexibility and creativity in predicting plausible futures. This is contrasted with current training methods that are rigid and require vast amounts of data. The idea of neural networks generating plausible images or futures is presented as a solution to overcome the limitations of supervised learning.

15:04

🏥 AI for Social Good: Personalized Medicine and Legal Services

In this section, the speaker turns to the potential of AI for societal benefit, such as in personalized medicine and providing affordable legal services. Deep learning could transform healthcare by offering personalized treatments, especially in underprivileged regions. The speaker emphasizes the importance of ethical AI, urging individuals to educate themselves on its implications. The speaker also highlights their personal choice to remain in academia to foster the next generation of deep learning experts, ensuring AI research is guided by a focus on public good, rather than just commercial interests.

Keywords

💡Artificial Intelligence (AI)

Artificial Intelligence (AI) refers to machines' ability to perform tasks that typically require human intelligence. In the video, AI is described as part of a new industrial revolution that expands human cognitive abilities, impacting both manual and mental labor. The discussion highlights AI's role in learning from data and how it is being applied to solve complex problems like playing Go or improving medical care.

💡Machine Learning

Machine Learning is a subset of AI that enables machines to learn from data and improve over time without being explicitly programmed. In the video, it is explained through examples like AlphaGo, which mastered the game of Go by analyzing millions of moves and playing against itself. This illustrates the process of machines learning from examples to develop skills autonomously.

💡AlphaGo

AlphaGo is a machine learning system created by DeepMind that used deep learning to beat the world champion in the complex board game Go. The video highlights AlphaGo as a major breakthrough in AI, showing how advanced machine learning algorithms can tackle tasks that were previously thought to be too complex for computers, such as mastering Go, which had been more difficult for computers than chess.

💡Deep Learning

Deep Learning is a type of machine learning inspired by the structure of the human brain, involving neural networks with multiple layers. The video discusses deep learning's application in areas such as speech recognition, image recognition, and natural language processing. These neural networks allow machines to learn hierarchical representations, mimicking the way human brains process visual and auditory information.

💡Supervised Learning

Supervised Learning is a machine learning approach where a computer learns from labeled data. The video describes how this method requires humans to provide answers to millions of questions, such as identifying objects in images. While powerful, supervised learning is limited because it requires extensive human input, unlike the more autonomous learning exhibited by humans or through unsupervised learning.

💡Unsupervised Learning

Unsupervised Learning allows machines to discover patterns in data without explicit guidance or labels. The video presents unsupervised learning as a key challenge in AI, crucial for solving complex tasks like intuitive physics or self-driving cars. Unlike supervised learning, it enables computers to understand and predict future scenarios by learning directly from raw data, similar to how children learn through observation.

💡Neural Networks

Neural Networks are computational systems inspired by the human brain, consisting of interconnected layers that process information. The video explains how neural networks have been key in advancing AI, especially through deep learning, which allows computers to transform data from raw pixels to high-level representations. This has enabled breakthroughs in tasks like image recognition and speech processing.

💡Representation Learning

Representation Learning is the process by which a machine learns to transform raw data into meaningful patterns or features. In the video, this concept is explained through examples of how neural networks learn different levels of representation, from pixels to more abstract concepts like objects or scenes. It is a fundamental aspect of deep learning, enabling machines to understand the world in a structured way.

💡Second Machine Age

The Second Machine Age refers to the ongoing revolution where machines are expanding human mental capabilities, not just physical ones. The video contrasts this with previous industrial revolutions that focused on enhancing mechanical power. AI and deep learning are the driving forces behind this age, as they allow machines to handle cognitive tasks that were once exclusively human, such as decision-making and problem-solving.

💡Personalized Medicine

Personalized Medicine is an approach to healthcare that tailors treatments to individual patients based on their unique data, such as genetic information. The video mentions personalized medicine as one of the potential social value applications of AI. As AI continues to advance, it could lead to improved healthcare by analyzing vast amounts of data to provide more precise and effective treatments for individuals.

Highlights

Artificial intelligence is leading to a new industrial revolution, expanding our cognitive abilities and mental power.

AlphaGo's victory over the world champion in Go showcased the power of machine learning and deep learning.

Machine learning allows computers to learn from data and examples, which is crucial for developing intelligence.

The challenge of programming computers with intuitive knowledge has been addressed by allowing machines to learn on their own.

Deep learning, inspired by the brain, has revolutionized AI, leading to advancements in speech recognition, computer vision, and language translation.

Deep neural networks can learn to transform low-level data (pixels) into high-level representations (abstract concepts).

Supervised learning, where computers learn from labeled data, has been very successful but has limitations compared to human learning.

Unsupervised learning, a key challenge for AI, involves discovering representations of data without explicit labels.

Unsupervised learning is crucial for applications like self-driving cars, where computers must predict and plan for future scenarios.

Deep learning models can generate new, plausible images, demonstrating creativity similar to human dreaming.

Montreal has become a hub for deep learning research, attracting many students and significant funding.

There is a high demand for deep learning experts, with many moving to industrial labs, though some, like the speaker, remain in academia.

Deep learning can be applied to socially valuable fields, such as personalized medicine and legal services.

The public should be informed about AI to participate in decisions about its future impact, both positive and negative.

Educational resources like papers, books, and online materials are available for those interested in learning about deep learning and AI.

Transcripts

Transcriber: Natasha Savic Reviewer: Claire Ghyselen

Our world is changing in many ways

and one of the things which is going to have a huge impact on our future

is artificial intelligence - AI,

bringing another industrial revolution.

Previous industrial revolutions expanded human's mechanical power.

This new revolution, this second machine age

is going to expand our cognitive abilities,

our mental power.

Computers are not just going to replace manual labor,

but also mental labor.

So, where do we stand today?

You may have heard about what happened last March

when a machine learning system called AlphaGo

used deep learning to beat the world champion at the game of Go.

Go is an ancient Chinese game

which had been much more difficult for computers to master

than the game of chess.

How did we succeed, now, after decades of AI research?

AlphaGo was trained to play Go.

First, by watching over and over

tens of millions of moves made by very strong human players.

Then, by playing against itself, millions of games.

Machine Learning allows computers to learn from examples.

To learn from data.

Machine learning has turned out to be a key

to cram knowledge into computers.

And this is important

because knowledge is what enables intelligence.

Putting knowledge into computers had been a challenge for previous approaches to AI.

Why?

There are many things which we know intuitively.

So we cannot communicate them verbally.

We do not have conscious access to that intuitive knowledge.

How can we program computers without knowledge?

What's the solution?

The solution is for machines to learn that knowledge by themselves,

just as we do.

And this is important because knowledge is what enables intelligence.

My mission has been to contribute to discover

and understand principles of intelligence through learning.

Whether animal, human or machine learning.

I and others believe that there are a few key principles,

just like the law of physics.

Simple principles which could explain our own intelligence

and help us build intelligent machines.

For example, think about the laws of aerodynamics

which are general enough to explain the flight of both, birds and planes.

Wouldn't it be amazing to discover such simple but powerful principles

that would explain intelligence itself?

Well, we've made some progress.

My collaborators and I have contributed in recent years in a revolution in AI

with our research on neural networks and deep learning,

an approach to machine learning which is inspired by the brain.

It started with speech recognition

on your phones, with neural networks since 2012.

Shortly after, came a breakthrough in computer vision.

Computers can now do a pretty good job of recognizing the content of images.

In fact, they approach human performance on some benchmarks over the last 5 years.

A computer can now get an intuitive understanding

of the visual appearance of a Go-board

that is comparable to that of the best human players.

More recently,

following some discoveries made in my lab,

deep learning has been used to translate from one language to another

and you are going to start seeing this in Google translate.

This is expanding the computer's ability

to understand and generate natural language.

But don't be fooled.

We are still very, very far from a machine

that would be as able as humans

to learn to master many aspects of our world.

So, let's take an example.

Even a two year old child is able to learn things

in a way that computers are not able to do right now.

A two year old child actually masters intuitive physics.

She knows when she drops a ball that it is going to fall down.

When she spills some liquids she expects the resulting mess.

Her parents do not need to teach her

about Newton's laws or differential equations.

She discovers all these things by herself in a unsupervised way.

Unsupervised learning actually remains one of the key challenges for AI.

And it may take several more decades of fundamental research

to crack that knot.

Unsupervised learning is actually trying to discover representations of the data.

Let me show you and example.

Consider a page on the screen that you're seeing with your eyes

or that the computer is seeing as an image, a bunch of pixels.

In order to answer a question about the content of the image

you need to understand its high-level meaning.

This high level meaning corresponds to the highest level of representation

in your brain.

Low down, you have the individual meaning of words

and even lower down, you have characters which make up the words.

Those characters could be rendered in different ways

with different strokes that make up the characters.

And those strokes are made up of edges

and those edges are made up of pixels.

So these are different levels of representation.

But the pixels are not sufficient by themselves

to make sense of the image,

to answer a high level question about the content of the page.

Your brain actually has these different levels of representation

starting with neurons in the first visual area of cortex - V1,

which recognizes edges.

And then, neurons in the second visual area of cortex - V2,

which recognizes strokes and small shapes.

Higher up, you have neurons which detect parts of objects

and then objects and full scenes.

Neural networks, when they're trained with images,

can actually discover these types of levels of representation

that match pretty well what we observe in the brain.

Both, biological neural networks, which are what you have in your brain

and the deep neural networks that we train on our machines

can learn to transform from one level of representation to the next,

with the high levels corresponding to more abstract notions.

For example the abstract notion of the character A

can be rendered in many different ways at the lowest levels

as many different configurations of pixels

depending on the position, rotation, font and so on.

So, how do we learn these high levels of representations?

One thing that has been very successful up to now

in the applications of deep learning,

is what we call supervised learning.

With supervised learning, the computer needs to be taken by the hand

and humans have to tell the computer the answer to many questions.

For example, on millions and millions of images, humans have to tell the machine

well... for this image, it is a cat.

For this image, it is a dog.

For this image, it is a laptop.

For this image, it is a keyboard, And so on, and so on millions of times.

This is very painful and we use crowdsourcing to manage to do that.

Although, this is very powerful

and we are able to solve many interesting problems,

humans are much stronger

and they can learn over many more different aspects of the world

in a much more autonomous way,

just as we've seen with the child learning about intuitive physics.

Unsupervised learning could also help us deal with self-driving cars.

Let me explain what I mean:

Unsupervised learning allows computers to project themselves into the future

to generate plausible futures conditioned on the current situation.

And that allows computers to reason and to plan ahead.

Even for circumstances they have not been trained on.

This is important

because if we use supervised learning we would have to tell the computers

about all the circumstances where the car could be

and how humans would react in that situation.

How did I learn to avoid dangerous driving behavior?

Did I have to die a thousand times in an accident?

(Laughter)

Well, that's the way we train machines right now.

So, it's not going to fly or at least not to drive.

(Laughter)

So, what we need is to train our models

to be able to generate plausible images or plausible futures, be creative.

And we are making progress with that.

So, we're training these deep neural networks

to go from high-level meaning to pixels

rather than from pixels to high level meaning,

going into the other direction through the levels of representation.

And this way, the computer can generate images

that are new images different from what the computer has seen

while it was trained,

but are plausible and look like natural images.

We can also use these models to dream up strange,

sometimes scary images,

just like our dreams and nightmares.

Here's some images that were synthesized by the computer

using these deep charted models.

They look like natural images

but if you look closely, you will see they are different

and they're still missing some of the important details

that we would recognize as natural.

About 10 years ago,

unsupervised learning has been a key to the breakthrough

that we obtained discovering deep learning.

This was happening in just few labs, including mine at the time

at a time when neural networks were not popular.

They were almost abandoned by the scientific community.

Now, things have changed a lot.

It has become a very hot field.

There are now hundreds of students every year applying for graduate studies

at my lab with my collaborators.

Montreal has become the largest academic concentration

of deep learning researchers in the world.

We just received a huge research grant of 94 million dollars

to push the boundaries of AI and data science

and also to transfer technology of deep learning and data science to the industry.

Business people stimulated by all this are creating start-ups, industrial labs,

many of which near the universities.

For example,

just a few weeks ago, we announced the launch of a start-up factory

called 'Element AI'

which is going to focus on the deep learning applications.

There are just not enough deep learning experts.

So, they are getting paid crazy salaries,

and many of my former academic colleagues have accepted generous deals

from companies to work in industrial labs.

I, for myself, have chosen to stay in university,

to work for the public good,

to work with students,

to remain independent.

To guide the next generation of deep learning experts.

One thing that we are doing beyond commercial value

is thinking about the social implications of AI.

Many of us are now starting to turn our eyes

towards social value added applications, like health.

We think that we can use deep learning

to improve treatment with personalized medicine.

I believe that in the future,

as we collect more data from millions and billions people around the earth,

we will be able to provide medical advice

to billions of people who don't have access to it right now.

And we can imagine many other applications for social value of AI.

For example, something that will come out of our research

on natural language understanding

is providing all kinds of services

like legal services, to those who can't afford them.

We are now turning our eyes

also towards the social implications of AI in my community.

But it's not just for experts to think about this.

I believe that beyond the math and the jargon,

ordinary people can get the sense

of what goes on under the hood

enough to participate in the important decisions

that will take place, in the next few years and decades about AI.

So please,

set aside your fees and give yourself some space to learn about it.

My collaborators and I have written several introductory papers

and a book entitled "Deep Learning"

to help students and engineers jump into this exciting field.

There are also many online resources: softwares, tutorials, videos..

and many undergraduate students are learning a lot of this

about research in deep learning by themselves,

to later join the ranks of labs like mine.

Ai is going to have a profound impact on our society.

So, it's important to ask: How are we going to use it?

Immense positives may come along with negatives

such as military use

or rapid disruptive changes in the job market.

To make sure the collective choices that will be made about AI

in the next few years,

will be for the benefit of all,

every citizen should take an active role

in defining how AI will shape our future.

Thank you.

(Applause)