How AI Will Step Off the Screen and into the Real World | Daniela Rus | TED
Summary
TLDRA robotics expert reflects on a key lesson learned during a student project—how the unpredictability of the physical world challenges digital systems. The speaker, now leading MIT's Computer Science and AI Lab, discusses the exciting future of combining AI with robotics. Introducing 'physical intelligence,' the fusion of AI's decision-making abilities with robots' physical capabilities, the talk explores how this breakthrough will transform industries, from designing robots using text prompts to creating adaptive AI. The speaker envisions a world where intelligent machines enhance human abilities, urging collaboration to shape the future of technology responsibly.
Takeaways
- 🤖 Robotics requires careful planning to match the robot's capabilities with the physical world, as shown by the mishap with the cake-cutting robot.
- 🎓 The speaker leads MIT's Computer Science and AI lab, where they're merging AI with robotics to bring intelligence into the physical world.
- 🧠 AI and robotics are traditionally separate fields, but now, AI is moving from digital decision-making into real-world physical applications.
- 🔧 Physical intelligence combines AI's decision-making with the mechanical prowess of robots to make machines smarter and more capable in the physical world.
- 💡 Current AI systems are large, error-prone, and frozen after training, while 'liquid networks' offer a more efficient, adaptive, and understandable solution.
- 🦠 Liquid networks are inspired by the simple neural structures of C. elegans, offering compact and explainable AI that continues to learn after deployment.
- 🚗 Liquid networks lead to better performance in tasks like self-driving, where they focus on relevant features, unlike traditional AI, which can be easily confused.
- 🔬 Text-to-robot and image-to-robot technology allows rapid prototyping, transforming digital designs into real-world machines with ease and precision.
- 👨🍳 Robots can learn tasks from humans by collecting physical data from sensors, making them capable of mimicking human actions with grace and agility.
- 🌍 The speaker emphasizes the potential of physical intelligence to amplify human capabilities, improve daily life, and help create a better future for humanity and the planet.
Q & A
What inspired the speaker's interest in robotics?
-The speaker's interest in robotics began during their time as a student when they and a group of friends attempted to program a robot to cut a birthday cake for their professor.
What was the lesson learned from the cake-cutting robot incident?
-The speaker learned that the physical world, with its laws of physics and imprecisions, is far more demanding than the digital world.
How does the speaker describe the current relationship between AI and robotics?
-Currently, AI and robotics are largely separate fields. AI focuses on decision-making and learning in the digital realm, while robots are physical machines that can execute pre-programmed tasks but lack intelligence.
What is 'physical intelligence' according to the speaker?
-'Physical intelligence' refers to the integration of AI's ability to understand data with the physical abilities of robots, enabling machines to think and interact intelligently in the real world.
What are some of the challenges in achieving physical intelligence?
-Challenges include shrinking AI to run on smaller devices, such as robots, and ensuring that AI can adapt to physical environments without making mistakes.
How do liquid networks improve traditional AI models?
-Liquid networks use fewer neurons with more complex mathematical functions, inspired by biological neurons like those in the C. elegans worm. These networks are adaptable and provide more explainable and focused decision-making compared to traditional AI.
What is the difference between traditional AI and liquid networks in adapting to real-world environments?
-Traditional AI systems are fixed after training and cannot adapt, while liquid networks continue to learn and adapt based on the input they receive, even after deployment.
How does the speaker's lab turn text prompts into robots?
-In the lab, a system starts with a language prompt, generates designs for a robot including its shape, materials, and control system, and then refines the design through simulation until it meets specifications.
What is the significance of human-to-robot learning in the speaker's research?
-Human-to-robot learning allows machines to learn tasks from human behavior, such as food preparation and cleaning, by using physical data like muscle movements and gaze patterns.
What are the potential future applications of physical intelligence?
-Physical intelligence could lead to personalized robots that assist in daily life, bespoke machines for work, and robots that can learn and perform tasks with agility and precision, extending human capabilities.
Outlines
🤖 A Cake-Cutting Robot Mishap and a Lesson in Robotics
The speaker shares a humorous anecdote about a project during their student days where a robot was programmed to cut a cake for their professor’s birthday. However, due to a lack of coordination, the robot, designed for a soft sponge cake, faced a square ice cream cake instead and failed spectacularly. Despite the failure, the professor appreciated the effort and referred to the incident as a ‘control singularity’—a robotics term. This experience taught the speaker that the physical world, with its unpredictable nature, poses far greater challenges than the digital realm. Now leading MIT's Computer Science and AI Lab, the speaker discusses how AI and robotics, previously separate fields, are converging, moving AI beyond digital screens and into the 3D physical world, thus enabling the next breakthrough—physical intelligence.
🚗 From Noisy AI to Streamlined Liquid Networks
This section contrasts traditional AI with liquid networks in the context of self-driving cars. The speaker highlights the inefficiencies of conventional AI, which relies on thousands of neurons that result in noisy, unfocused attention maps, causing the AI to misread its surroundings. In contrast, liquid networks, with only 19 neurons, provide a cleaner, more focused decision-making process. The speaker explains how traditional AI neurons are simple on/off units that struggle with complex real-world tasks, while liquid networks use differential equations inspired by the neural structure of worms, allowing for adaptability and continued learning after deployment. This innovation dramatically enhances AI's performance in dynamic environments, such as seasonal changes when navigating a forest.
🐰 Creating Robots from Text and Images
The speaker introduces the concept of transforming text and images into functional robots through a new AI-guided design process. By using physical constraints and simulations, robots can be designed from prompts like ‘make me a robot that can walk forward,’ and the system generates the required specifications for its construction. The speaker describes a specific example where an image of a bunny was turned into a physical, robotic version through 3D modeling and printing. This process dramatically speeds up prototyping and testing, leading to faster innovation cycles in robotics design and manufacturing.
👨🍳 Teaching Robots Human Tasks through Physical Intelligence
In this section, the speaker discusses how robots can be taught human tasks by observing physical data such as muscle movements, posture, and gaze. The speaker’s lab has created a kitchen environment to capture this data and train AI systems to replicate these tasks with robots. This approach allows machines to move with greater agility and grace, adapting to new situations. By utilizing liquid networks and physical intelligence, these machines can learn from human actions and become more versatile in completing tasks like cooking and cleaning, showcasing the far-reaching potential of physical intelligence in everyday life.
🌍 The Potential and Responsibility of Physical Intelligence
The speaker concludes by emphasizing the vast potential of physical intelligence, where AI-powered machines can transcend human limitations, assist with daily tasks, and innovate across industries. However, the speaker also underscores the importance of guiding AI responsibly, noting that current AI systems are unsustainable and prone to errors. By developing AI that interacts with the physical world and continues to learn, humanity can unlock unprecedented opportunities while ensuring a better future for both people and the planet. The speaker invites the audience to take part in shaping this future, either by developing, using, or inventing with physical intelligence.
Mindmap
Keywords
💡Physical intelligence
💡AI (Artificial Intelligence)
💡Liquid networks
💡Robotics
💡C. elegans
💡Control singularity
💡Self-driving car
💡Differential equations
💡Text-to-robot
💡Learning from humans
Highlights
A group of students attempted to program a robot to cut a cake for their professor's birthday, but the robot malfunctioned due to a mismatch between the soft cake they planned for and the hard ice cream cake they received.
Despite the failure, the professor praised the malfunction as a 'control singularity,' highlighting the unpredictable nature of robotics in real-world applications.
The speaker leads MIT's Computer Science and AI Lab, which focuses on combining AI with robotics to develop intelligent machines capable of operating in the physical world.
The current separation between AI and robotics is beginning to dissolve, as AI is transitioning from computer screens into the real world, paving the way for 'physical intelligence.'
'Physical intelligence' is defined as the combination of AI's ability to understand data with the mechanical execution of robots, allowing machines to perform tasks more intelligently.
For AI to integrate into the physical world, it must run on smaller, more efficient computers, enabling robots to carry out tasks based on real-time data.
The concept of 'liquid networks' is introduced as a new AI model inspired by the neurons of the worm C. elegans, making AI solutions more compact and understandable.
Traditional AI models rely on thousands of neurons to make decisions, but liquid networks use only 19 neurons, resulting in a cleaner, more focused decision-making process.
Liquid networks adapt to new data after training, unlike traditional AI systems which are frozen and cannot improve once deployed.
The speaker demonstrates how liquid networks outperform traditional AI in tasks such as identifying objects in different environments, like woods during summer and fall.
Liquid networks are not just limited to AI software; they are also being integrated into robotics, creating adaptable machines that can learn from humans.
The lab has developed systems that can design robots based on text or image prompts, drastically reducing the time required for prototyping and testing new machines.
The future of AI and robotics will involve creating machines that not only learn from humans but also perform tasks in physical spaces like food preparation and cleaning.
Physical intelligence enables machines to move with grace and agility, adapting to real-world situations and interacting with humans more seamlessly.
AI's future lies in its ability to extend human capabilities, amplify strengths, and refine precision, allowing us to build advanced tools that improve our interaction with the world.
The speaker calls for human guidance over AI development, emphasizing the responsibility we have to ensure technology benefits humanity and the planet.
Transcripts
When I was a student studying robotics,
a group of us decided to make a present for our professor's birthday.
We wanted to program our robot to cut a slice of cake for him.
We pulled an all-nighter writing the software,
and the next day, disaster.
We programmed this robot to cut a soft, round sponge cake,
but we didn't coordinate well.
And instead, we received a square hard ice cream cake.
The robot flailed wildly and nearly destroyed the cake.
(Laughter)
Our professor was delighted, anyway.
He calmly pushed the stop button
and declared the erratic behavior of the robot
a control singularity.
A robotics technical term.
I was disappointed, but I learned a very important lesson.
The physical world,
with its physics laws and imprecisions,
is a far more demanding space than the digital world.
Today, I lead MIT's Computer Science and AI lab,
the largest research unit at MIT.
This is our building, where I work with brilliant and brave researchers
to invent the future of computing and intelligent machines.
Today in computing,
artificial intelligence and robotics are largely separate fields.
AI has amazed you with its decision-making and learning,
but it remains confined inside computers.
Robots have a physical presence and can execute pre-programmed tasks,
but they're not intelligent.
Well, this separation is starting to change.
AI is about to break free from the 2D computer screen interactions
and enter a vibrant, physical 3D world.
In my lab, we're fusing the digital intelligence of AI
with the mechanical prowess of robots.
Moving AI from the digital world into the physical world
is making machines intelligent
and leading to the next great breakthrough,
what I call physical intelligence.
Physical intelligence is when AI's power to understand text,
images and other online information
is used to make real-world machines smarter.
This means AI can help pre-programmed robots do their tasks better
by using knowledge from data.
With physical intelligence,
AI doesn't just reside in our computers,
but walks, rolls, flies
and interacts with us in surprising ways.
Imagine being surrounded by helpful robots at the supermarket.
The one on the left can help you carry a heavy box.
To make it happen, we need to do a few things.
We need to rethink how machines think.
We need to reorganize how they are designed and how they learn.
So for physical intelligence,
AI has to run on computers that fit on the body of the robot.
For example, our soft robot fish.
Today's AI uses server farms that do not fit.
Today's AI also makes mistakes.
This AI system on a robot car does not detect pedestrians.
For physical intelligence,
we need small brains that do not make mistakes.
We're tackling these challenges using inspiration
from a worm called C. elegans
In sharp contrast to the billions of neurons in the human brain,
C. elegans has a happy life on only 302 neurons,
and biologists understand the math of what each of these neurons do.
So here's the idea.
Can we build AI using inspiration from the math of these neurons?
We have developed, together with my collaborators and students,
a new approach to AI we call “liquid networks.”
And liquid networks results in much more compact
and explainable solutions than today's traditional AI solutions.
Let me show you.
This is our self-driving car.
It's trained using a traditional AI solution,
the kind you find in many applications today.
This is the dashboard of the car.
In the lower right corner, you'll see the map.
In the upper left corner, the camera input stream.
And the big box in the middle with the blinking lights
is the decision-making engine.
It consists of tens of thousands of artificial neurons,
and it decides how the car should steer.
It is impossible to correlate the activity of these neurons
with the behavior of the car.
Moreover, if you look at the lower left side,
you see where in the image this decision-making engine looks
to tell the car what to do.
And you see how noisy it is.
And this car drives by looking at the bushes and the trees
on the side of the road.
That's not how we drive.
People look at the road.
Now contrast this with our liquid network solution,
which consists of only 19 neurons rather than tens of thousands.
And look at its attention map.
It's so clean and focused on the road horizon
and the side of the road.
Because these models are so much smaller,
we actually understand how they make decisions.
So how did we get this performance?
Well, in a traditional AI system,
the computational neuron is the artificial neuron,
and the artificial neuron is essentially an on/off computational unit.
It takes in some numbers, adds them up,
applies some basic math
and passes along the result.
And this is complex
because it happens across thousands of computational units.
In liquid networks,
we have fewer neurons,
but each one does more complex math.
Here's what happens inside our liquid neuron.
We use differential equations to model the neural computation
and the artificial synapse.
And these differential equations
are what biologists have mapped for the neural structure of the worms.
We also wire the neurons differently to increase the information flow.
Well, these changes yield phenomenal results.
Traditional AI systems are frozen after training.
That means they cannot continue to improve
when we deploy them in a physical world in the wild.
We just wait for the next release.
Because of what's happening inside the liquid neuron,
liquid networks continue to adapt after training
based on the inputs that they see.
Let me show you.
We trained traditional AI and liquid networks
using summertime videos like these ones,
and the task was to find things in the woods.
All the models learned how to do the task in the summer.
Then we tried to use the models on drones in the fall.
The traditional AI solution gets confused by the background.
Look at the attention map, cannot do the task.
Liquid networks do not get confused by the background
and very successfully execute the task.
So this is it.
This is the step forward:
AI that adapts after training.
Liquid networks are important
because they give us a new way of getting machines to think
that is rooted into physics models,
a new technology for AI.
We can run them on smartphones, on robots,
on enterprise computers,
and even on new types of machines
that we can now begin to imagine and design.
The second aspect of physical intelligence.
So by now you've probably generated images using text-to-image systems.
We can also do text-to-robot,
but not using today's AI solutions because they work on statistics
and do not understand physics.
In my lab,
we developed an approach that guides the design process
by checking and simulating the physical constraints for the machine.
We start with a language prompt,
"Make me a robot that can walk forward,"
and our system generates the designs including shape, materials, actuators,
sensors, the program to control it
and the fabrication files to make it.
And then the designs get refined in simulation
until they meet the specifications.
So in a few hours we can go from idea
to controllable physical machine.
We can also do image-to-robot.
This photo can be transformed into a cuddly robotic bunny.
To do so, our algorithm computes a 3D representation of the photo
that gets sliced and folded, printed.
Then we fold the printed layers, we string some motors and sensors.
We write some code, and we get the bunny you see in this video.
We can use this approach to make anything almost,
from an image, from a photo.
So the ability to transform text into images
and to transform images into robots is important,
because we are drastically reducing the amount of time
and the resources needed to prototype and test new products,
and this is allowing for a much faster innovation cycle.
And now we are ready to even make the leap
to get these machines to learn.
The third aspect of physical intelligence.
These machines can learn from humans how to do tasks.
You can think of it as human-to-robot.
In my lab, we created a kitchen environment
where we instrument people with sensors,
and we collect a lot of data about how people do kitchen tasks.
We need physical data
because videos do not capture the dynamics of the task.
So we collect muscle, pose, even gaze information
about how people do tasks.
And then we train AI using this data
to teach robots how to do the same tasks.
And the end result is machines that move with grace and agility,
as well as adapt and learn.
Physical intelligence.
We can use this approach to teach robots
how to do a wide range of tasks:
food preparation, cleaning and so much more.
The ability to turn images and text into functional machines,
coupled with using liquid networks
to create powerful brains for these machines
that can learn from humans, is incredibly exciting.
Because this means we can make almost anything we imagine.
Today's AI has a ceiling.
It requires server farms.
It's not sustainable.
It makes inexplicable mistakes.
Let's not settle for the current offering.
When AI moves into the physical world,
the opportunities for benefits and for breakthroughs is extraordinary.
You can get personal assistants that optimize your routines
and anticipate your needs,
bespoke machines that help you at work
and robots that delight you in your spare time.
The promise of physical intelligence is to transcend our human limitations
with capabilities that extend our reach,
amplify our strengths
and refine our precision
and grant us ways to interact with the world
we've only dreamed of.
We are the only species so advanced, so aware,
so capable of building these extraordinary tools.
Yet, developing physical intelligence
is teaching us that we have so much more to learn
about technology and about ourselves.
We need human guiding hands over AI sooner rather than later.
After all, we remain responsible for this planet
and everything living on it.
I remain convinced that we have the power
to use physical intelligence to ensure a better future for humanity
and for the planet.
And I'd like to invite you to help us in this quest.
Some of you will help develop physical intelligence.
Some of you will use it.
And some of you will invent the future.
Thank you.
(Applause)
関連動画をさらに表示
Next Up for AI? Dancing Robots | Catie Cuan | TED
The Rapid Evolution of AI in the Warehouse
OpenAI CEO on the future of programming | Sam Altman and Lex Fridman
Elon Musk - Where AI's Biggest Impact Will Be
World Changing: Data Science and AI | Fred Blackburn | TEDxWakeForestU
What is Mechatronics Engineering? || All you need to know
5.0 / 5 (0 votes)