Face-ing Our Realistic Digital Self | Koki Nagano | TEDxCharlottesville

Summary

In this talk, the speaker explores the intersection of computer graphics, artificial intelligence, and human interaction. From a childhood fascination with realism in drawing to cutting-edge research on 3D digital avatars, the speaker shares insights on creating lifelike digital representations that can engage in real-time, face-to-face communication. Highlighting advancements like deepfake technology and realistic facial simulations, the presentation delves into the potential of digital avatars in entertainment, fashion, and history preservation, while emphasizing the importance of ethical considerations in the future of these technologies.

Takeaways

• 😀 The fascination between drawing and photography inspired the speaker to explore how to create hyper-realistic digital avatars and objects using computer graphics and AI.
• 😀 The speaker's research focuses on using modern photography to create computer-generated images of real-world objects to understand and replicate realistic human communication.
• 😀 Virtual telepresence, where digital versions of people interact in lifelike ways, could revolutionize face-to-face communication, allowing people to meet virtually as if they were physically together.
• 😀 The USC Shoah Foundation project aimed to create a 3D visual archive of Holocaust survivors, preserving their testimonies and enabling future generations to engage in lifelike conversations with digital versions of them.
• 😀 High-resolution cameras and advanced computer systems are key to capturing lifelike digital representations of faces for use in films, virtual assistants, and digital avatars.
• 😀 The use of 3D capture techniques and specialized equipment (like the Light Stage at USC) allows for the creation of photo-realistic digital faces and characters, crucial for creating believable avatars in films and virtual experiences.
• 😀 Creating a realistic digital human face requires capturing intricate skin details that normal cameras can't perceive, including pores, wrinkles, and microstructures.
• 😀 The development of deep fake technology has made it possible to create realistic face-swapping videos with just a few days of training and a vast amount of online images, democratizing the creation of lifelike avatars.
• 😀 New machine learning technologies, like Pin Screen, allow for the creation of realistic 3D digital faces from a single picture, which can mimic facial expressions and head movements in real-time.
• 😀 Digital avatars have vast potential for industries like fashion, where users could digitally preview how clothing fits their virtual selves before making purchases, offering a more immersive shopping experience.
• 😀 While digital avatars offer exciting possibilities, the potential for misuse, such as deep fake videos, underscores the importance of collaboration between researchers, tech developers, and policymakers to ensure ethical usage of these technologies.

Q & A

What inspired the speaker to pursue research in computer graphics and artificial intelligence?

The speaker was inspired by a childhood fascination with drawing and how to add realism to drawings that could surpass photographs. This curiosity led them to explore the intersection of computer graphics and artificial intelligence, aiming to create virtual humans that can be believed as real people.

What is the significance of human face-to-face communication in the speaker's research?

The speaker emphasizes the importance of facial expressions in human communication, as it is a fundamental form of interaction that transcends cultures. They aim to recreate this experience in virtual environments, allowing lifelike digital avatars to engage in face-to-face communication.

What was the goal of the project with USC Shoah Foundation?

The goal of the project was to create a 3D visual archive of Holocaust survivors, allowing future generations to have lifelike conversations with digital versions of survivors' testimonies, preserving their stories for posterity.

How did the speaker's team capture realistic facial expressions and gestures for their 3D avatars?

The team used a special camera array to capture interviews and gestures from all angles, including hand gestures and eye gaze. They also developed a 3D display system to show these interactions from multiple perspectives, ensuring that viewers could experience the conversations as lifelike and interactive.

What role does conversational AI play in the speaker's research?

Conversational AI, similar to systems like Siri or Alexa, is integrated into the research to enable digital avatars to interact with users in a natural and engaging manner, responding to questions and engaging in real-time conversations.

How does the speaker's research relate to the use of digital avatars in movies?

The research draws from techniques used in movies, such as CGI, to create realistic digital avatars. For example, the use of CGI to digitally recreate Paul Walker's character in *Furious 7* after his passing showcases the high fidelity required for realistic digital humans in film.

What is 3D capture, and how does it contribute to creating realistic digital humans?

3D capture involves taking multiple high-resolution images from different lighting conditions to create accurate 3D models of faces. This technique helps produce lifelike avatars, and at USC, the 'light stage' device is used to achieve these results by capturing detailed facial features in 3D.

What is the challenge in creating realistic digital faces for close-up shots in movies?

The challenge lies in capturing minute details of human skin, such as ridges, grooves, and pores. To ensure realistic close-ups, new simulation algorithms are used to synthesize these tiny details, making digital faces appear lifelike even under close scrutiny.

What is deepfake technology, and how does it relate to the speaker's research?

Deepfake technology uses deep learning algorithms to create fake videos where individuals appear to say or do things they never did. The speaker's research focuses on how this technology can be used to create digital avatars, with the potential for non-technical users to create realistic avatars and face-swapping videos at no cost.

What are the potential applications of the speaker's technology in the fashion industry?

The technology could revolutionize the fashion industry by allowing consumers to digitally try on clothes using realistic 3D avatars before purchasing. This can enhance online shopping experiences by providing a more personalized and accurate preview of how clothing fits and looks.