ZLUDA: CUDA For AMD GPUs Returns From The Grave

Summary

The video discusses the evolution of ZLUDA, an open-source project aimed at enabling CUDA support for non-Nvidia GPUs, particularly AMD and Intel. It explains how ZLUDA began as a low-level hardware access tool for Intel GPUs and later received AMD funding to build CUDA compatibility on ROCm. Despite promising performance, AMD retracted its support in 2024, causing the project to be temporarily taken down. The creator is now rebuilding ZLUDA, focusing on machine learning workloads and broader GPU architecture support, with future development funded by an unknown entity.

Takeaways

• 🚀 The speaker is relieved they didn't make a video earlier because the project has evolved a lot.
• 💻 The project, initially known as ZLUDA, started in 2020 as an open-source CUDA support for Intel GPUs.
• 🔧 ZLUDA provides low-level hardware access, allowing CUDA applications to run on Intel GPUs, with limitations.
• 📉 While not perfect, benchmarks showed promising results, although performance was generally slower than Nvidia GPUs.
• 🔴 In 2024, AMD quietly funded a CUDA implementation built on ROCm, offering a drop-in solution for Nvidia CUDA apps.
• 🛑 In August 2024, AMD requested the removal of the ZLUDA code from public repositories, citing legal issues.
• 🔄 ZLUDA has entered its third phase, with development resuming and a commercial organization backing it.
• 🤖 Machine learning workloads will be prioritized in the new version of ZLUDA, targeting frameworks like PyTorch and TensorFlow.
• 📉 Ray tracing support has been dropped due to performance issues and high maintenance costs.
• 🧩 The new version of ZLUDA will be open-source, focusing on improving code quality and GPU support.

Q & A

What is the ZLUDA project and how did it start?

ZLUDA is an open-source project designed to provide drop-in CUDA support for Intel Xe/UHD graphics. It began as a project built on Intel's oneAPI Level Zero API, enabling CUDA applications to run on Intel GPUs with near-native performance.

What is the primary goal of Intel's oneAPI Level Zero API?

The primary goal of Intel's oneAPI Level Zero API is to offer low-level hardware access for accelerated devices, providing direct-to-metal interfaces for offloading tasks to these devices. It supports a broader set of language features like function pointers and unified memory.

What was significant about ZLUDA's ability to run CUDA applications?

The key significance was that ZLUDA could run unmodified CUDA applications on Intel GPUs. This meant developers didn’t need to alter their code, making it easier to run existing CUDA applications on non-Nvidia hardware, though with some limitations.

How did AMD become involved in the ZLUDA project?

AMD quietly funded a similar project to ZLUDA, aiming to create a drop-in CUDA implementation built on AMD's ROCm. This allowed many Nvidia CUDA applications to run on AMD's ROCm stack without needing source code adaptation.

What were some of the challenges and limitations of AMD's CUDA implementation?

While promising, AMD's CUDA implementation was slower than Nvidia's in many cases and was still in the experimental stage. Although competitive in some benchmarks, it was not yet a complete replacement for Nvidia GPUs in high-performance environments.

Why was ZLUDA taken down from GitHub in August 2024?

The ZLUDA code was taken down from GitHub at AMD's request. Although AMD initially approved the release, its legal department later argued that the release was not legally binding, leading to the rollback.

What legal precedent was mentioned regarding the open-source legality of ZLUDA?

The Oracle vs. Google case concerning Java was mentioned as a legal precedent, supporting the idea that the ZLUDA project was protected under law despite the code being retracted at AMD's request.

What is the current state of the ZLUDA project as of October 2024?

As of October 2024, ZLUDA is entering its 'third life' with funding from an unknown commercial organization. The developer has rolled back the code to its pre-AMD state and is focusing on rebuilding and improving it, with a goal to have a functional version by Q3 2025.

What is the future focus of the ZLUDA project?

The future focus of ZLUDA will be on machine learning workloads, as these are in high demand. The project aims to support frameworks like PyTorch, TensorFlow, and Llama.cpp, moving away from creator-focused applications and ray tracing.

How will the new ZLUDA differ from its previous version?

The new ZLUDA will target AMD GPUs, specifically RDNA1 and newer models, and focus on machine learning workloads. It will no longer support pre-RDNA1 architectures or complex features like ray tracing. Additionally, Windows support will be more limited and less user-friendly.