Code with me: Machine learning on a Macbook GPU (works for all M1, M2, M3) for a 10x speedup

Summary

This video demonstrates how to adapt machine learning models from using NVIDIA CUDA to running efficiently on Apple's M1, M2, or M3 chips with Metal Performance Shaders (MPS). The presenter explains common issues when running models on Mac GPUs and provides simple code adjustments to switch from CUDA to MPS for training, prediction, and pre-trained model loading. Through three coding examples, the video highlights PyTorch compatibility with MPS, resolving device-related errors, and optimizing data types, allowing models to run up to 20 times faster on Apple Silicon. Ideal for experienced programmers working on machine learning projects.

Takeaways

• 💻 Apple's M1, M2, and M3 chips offer significant speed improvements for machine learning tasks, up to 10-20 times faster than traditional CPUs.
• 🚀 Most machine learning models are programmed for NVIDIA GPUs using the CUDA framework, which is not natively supported on Macs with Apple Silicon.
• 🛠️ Adapting existing models to run on Mac's MPS (Metal Performance Shaders) requires minor code adjustments.
• 👨‍💻 The presenter assumes viewers have programming experience and will be demonstrating solutions quickly.
• 🔍 The first example uses a U-Net model for medical image segmentation, showing how to adjust code for MPS support.
• 🔧 Code modifications include changing device settings and handling pre-trained model loading with adjustments to the `torch.load` function.
• 📚 The second example focuses on a U-Net model in a Jupyter notebook, emphasizing the need to specify the correct backend and data types for MPS.
• 🔄 The third example deals with the CLIP model for image encoding, highlighting the importance of ensuring both model and input data are on the same device.
• 💾 Errors encountered during the process are used as a diagnostic tool to guide further code adjustments.
• 📈 The script provides a step-by-step guide to troubleshoot and optimize machine learning models for Apple Silicon GPUs.

Q & A

What is the main challenge with running machine learning models on Apple Silicon Macs with M1, M2, or M3 chips?

The main challenge is that most machine learning models are programmed for NVIDIA GPUs using the CUDA framework, which doesn't run natively on Apple Silicon Macs. As a result, these models either produce errors or default to running on the CPU, leading to slower performance.

How can you adapt existing machine learning models to run on Apple Silicon GPUs?

You can adapt machine learning models to run on Apple Silicon GPUs by making simple changes in the code to utilize the MPS (Metal Performance Shaders) framework, which is Apple's version of GPU acceleration. These changes involve specifying the correct device in the code, such as replacing references to CUDA with MPS.

What is MPS, and how does it relate to running machine learning models on Macs?

MPS stands for Metal Performance Shaders, which is Apple's framework for GPU acceleration. It allows machine learning models to leverage the GPU on Apple Silicon Macs, providing faster performance compared to running models on the CPU.

What are some examples of machine learning tasks mentioned in the script that benefit from using the MPS framework on Macs?

Examples mentioned include training, predicting, checkpoint loading, and compiling machine learning models. By utilizing the MPS framework, these tasks can see a significant speedup, potentially 10 to 20 times faster than using the CPU.

What specific change should be made to ensure a machine learning model runs on the Mac's MPS device instead of the CPU?

In the code, where the device is defined (typically set to CUDA for NVIDIA GPUs), you need to modify it to specify MPS. This change ensures that the model uses the Mac's GPU for faster performance.

How does PyTorch support the MPS framework, and since when has it been available?

PyTorch has supported the MPS framework since version 1.12, which was released a year or two ago. This support allows machine learning models to be adapted to run on Apple Silicon Macs using MPS.

What is the purpose of modifying the map location in the code when loading pre-trained models?

The map location must be modified to ensure that the pre-trained model is loaded onto the correct device (MPS, instead of CUDA). This change prevents errors that occur when trying to load a model onto a device that isn’t available.

What types of errors are commonly encountered when adapting machine learning models to run on Apple Silicon, and how can they be resolved?

Common errors include device mismatch (e.g., trying to load models on CUDA when it's not available) and data type issues. These can be resolved by ensuring the correct device is specified (MPS) and adjusting data types, such as casting integers to floats where necessary.

What is CLIP, and how is it used in the context of this script?

CLIP is a set of pre-trained models from OpenAI designed to align image encodings and text encodings, commonly used in generative image-to-text models. In the script, CLIP's image encoding capability is used as an example to demonstrate how to adapt pre-trained models to run on Apple's MPS framework.

What final adjustments are necessary to ensure inputs and models are running correctly on the MPS framework?

To ensure everything runs correctly on the MPS framework, you need to place both the inputs and the model on the MPS device. This step prevents errors that occur when variables are distributed across different devices.