Cybathlon - Legged Mobile Assistance for Quadriplegics

Summary

The video introduces a quadrupedal robot, equipped with a 'dino arm,' controlled by a quadriplegic pilot using a mouth joystick. The robot performs complex manipulation tasks like retrieving items from a mailbox and autonomously assisting with personal care, such as brushing teeth. Its control system includes a model-based whole-body controller for precise manipulation and a learned locomotion policy for walking, trained using reinforcement learning. The pilot seamlessly switches between control modes for movement and manipulation, achieving high precision after 15 hours of training across five sessions.

Takeaways

• 🤖 The system involves a quadrupedal robot with a dino arm mounted for manipulation tasks.
• 📦 The robot can perform tasks like opening a mailbox and retrieving packages.
• 👨‍🦽 The pilot, Samiun, is a quadriplegic who operates the robot using a mouth joystick.
• 🛠️ The robot is versatile and can perform complex tasks, such as picking up a scarf.
• 🪥 Autonomous functions include detecting the pilot's face and bringing a toothbrush to his mouth.
• 🎮 Two controllers are used: a model-based whole-body controller for manipulation and a learned locomotion policy for walking.
• 💡 The locomotion policy was trained in simulation using reinforcement learning.
• 🕹️ The pilot can switch between manipulation and locomotion modes using the mouth joystick.
• ⏱️ Intuitive control allows the pilot to achieve high proficiency within a short training period.
• 🎯 After 15 hours of training, the pilot demonstrated precise manipulation and synchronized movement of the wheelchair and robot.

Q & A

What type of robot is described in the script?

The robot is a quadrupedal robot body with a dino arm mounted on top for manipulation tasks.

How does the robot retrieve a package from the mailbox?

The robot uses its manipulator arm to open the mailbox and retrieve the package inside.

Who operates the robot in the scenario provided, and how?

A pilot named Samiun, who is quadriplegic, operates the robot using a mouth joystick.

What tasks can the robot perform besides retrieving a package?

The robot can perform complex manipulation tasks such as picking up a scarf and autonomous functions like bringing a toothbrush to the pilot's mouth.

What is the role of the mouth joystick in controlling the robot?

The mouth joystick allows the pilot to switch between manipulation and locomotion modes, providing an intuitive control scheme.

How are the robot’s manipulation tasks controlled?

The robot's manipulation tasks are controlled using a model-based whole body controller, which ensures high end-effector tracking accuracy and stable base pose.

What method is used to control the robot’s locomotion?

The robot's locomotion is controlled by a learned locomotion policy that was trained using reinforcement learning in a simulation environment.

How long did it take the pilot to achieve proficiency in operating the robot?

The pilot achieved proficiency in approximately 15 hours of training spread over five sessions.

What level of control precision did the pilot achieve after training?

The pilot achieved centimeter-precise manipulation ability and the ability to synchronize the movement of both the wheelchair and the robot.

What autonomous function is demonstrated by the robot in the script?

The robot demonstrates an autonomous function by detecting the face of the pilot and bringing a toothbrush into contact with his mouth.