The future of RenderMan & USD - Pixar CTO Steve May interview - SIGGRAPH 2022

Summary

Steve May, Pixar's CTO, discusses technological collaborations within Disney, highlighting the development of a machine learning-based denoiser that significantly reduced render times. He emphasizes Pixar's commitment to sharing innovations like RenderMan, which now includes the advanced denoiser. May also touches on the potential of Universal Scene Description (USD) in creating complex 3D environments beyond film, noting its adaptability and community-driven growth.

Takeaways

• 🎬 Steve May is Pixar's CTO with 24 years of experience, half of which involved working on films as a supervisor.
• 🤝 Pixar collaborates with other Disney studios, including ILM, Disney Animation, Marvel, and Disney Research Studios, sharing and developing technologies together.
• 🔍 A machine learning-based denoiser was developed in collaboration with Disney Research, improving upon an earlier version from Disney Animation, and first used internally on 'Toy Story 4'.
• ⏰ The new denoiser significantly reduced compute time for rendering frames by 75%, allowing for a quarter of the rendering time to be spent.
• 📈 Pixar has been refining and improving the denoiser over several years and is now including it with RenderMan for broader access.
• 🖼️ The denoiser is capable of producing high-quality images from a small number of render samples, enhancing images that initially appear as random pixels.
• 🤔 Discussions with partners like Disney Research and other studios determine what technologies to share externally, such as open-source or commercial versions of RenderMan.
• 🛠️ Transitioning technology to production involves generalizing and cleaning up the user interface to create a sellable software product.
• 🚀 Pixar is focusing on replacing RIS with a new XPU renderer, which is a complex process requiring time to ensure robustness for real production use.
• 🌐 Pixar's Universal Scene Description (USD) is a foundational technology for complex 3D scenes, with potential applications in the Metaverse and beyond.
• 🔄 USD supports layering, allowing for efficient manipulation of scenes with many simultaneous users, which is beneficial for both film production and virtual world creation.

Q & A

How long has Steve May been working at Pixar?

Steve May has been at Pixar for 24 years.

What is the role of Steve May at Pixar?

Steve May is the Chief Technology Officer (CTO) at Pixar.

What is the significance of the collaboration between Pixar and other Disney studios?

The collaboration allows for the sharing of technology and innovation among Pixar, ILM, Disney Animation, Marvel, and Disney Research Studios.

What was the purpose of developing a machine learning-based denoiser at Disney Research Studios?

The new machine learning-based denoiser was developed to address the denoising problem and to take the technology to the next level for film production.

How did the new denoiser impact the rendering process in 'Toy Story 4'?

The new denoiser reduced the compute time of frames by 75%, allowing for a quarter of the rendering time to be spent.

What is RenderMan, and why is the denoiser being included with it?

RenderMan is a high-end rendering software used in film production. The denoiser is being included to provide users with access to this advanced technology that can significantly improve rendering efficiency.

What does Steve May mean by 'pushing the knob very far' in the context of the denoiser?

It refers to testing the denoiser's capabilities to the extreme by using a minimal number of render samples to see how refined an image it can produce.

What is the current focus of Pixar's rendering team?

The team is focused on replacing RIS with the new XPU renderer, which is a more sophisticated and production-ready tool.

What is USD, and how does it relate to Pixar's work?

USD, or Universal Scene Description, is a system developed at Pixar for describing complex 3D scenes and is used for creating rich 3D worlds and characters for their movies.

How does Steve May view the adoption of USD beyond the film industry?

Steve May finds it exciting that USD has gained traction in other industries, such as construction and manufacturing, for optimizing digital twins and virtual environments.

What is Omniverse, and how does it relate to USD?

Omniverse is a platform built by NVIDIA on top of USD, which enables various industries to use USD for solving complex problems and creating virtual environments.

How does Pixar engage with the community and incorporate external contributions into USD?

Pixar engages through forums, working groups within the Academy Software Foundation, and direct interactions with companies like NVIDIA and Apple. They aim to support external contributions and are planning to expand the team to better manage community requests and develop the software.

What is the process Pixar uses to evaluate and adopt new technologies from events like SIGGRAPH?

Pixar uses a process called 'Rewind' where attendees of SIGGRAPH share exciting or interesting findings, followed by discussions and evaluations over time to determine if the technology is worth investing in.