The Web Neural Network (WebNN) API: Where we are and what's Next

Summary

In this video, Rob Kochman from Google Chrome and Rafael Cintron from Microsoft Edge introduce the WebNN API, an upcoming web standard designed to optimize neural network execution in web browsers. The WebNN API enables developers to seamlessly leverage hardware accelerators like CPUs, GPUs, and NPUs for AI computations without compromising the advantages of the web, such as accessibility and ease of use. The speakers discuss the current state of WebNN, its integration with popular frameworks like ONNX and TensorFlow, and future improvements aimed at enhancing device selection, interop with WebGPU, and overall performance. The demo showcases AI applications running directly in the browser, demonstrating the potential for cutting-edge AI on the web.

Takeaways

• 😀 WebNN is an upcoming web standard API aimed at improving neural network inference in web browsers.
• 😀 WebNN enables developers to access hardware accelerators like CPUs, GPUs, and NPUs for high-performance machine learning tasks.
• 😀 CPUs are great for universal reach and smaller workloads, while GPUs offer more power and efficiency for complex AI tasks.
• 😀 NPUs (Neural Processing Units) are becoming more prevalent and efficient, especially for long-running tasks where power consumption is crucial.
• 😀 WebNN aims to provide a unified API for accessing hardware resources, making it easier for developers to tap into the power of NPUs, GPUs, and CPUs.
• 😀 The WebNN API is designed to work seamlessly across multiple platforms and browsers, including Chrome and Edge.
• 😀 Developers can use popular ML frameworks like ONNX, TensorFlow, and PyTorch with WebNN, bringing their own models and frameworks.
• 😀 The API is currently available behind a flag in both Chrome and Edge, with an origin trial planned for developers to provide feedback.
• 😀 WebNN improves the efficiency of running machine learning models in the browser by avoiding the need for vendor-specific SDKs.
• 😀 WebNN supports hardware integration for various platforms, including CoreML for Apple devices and LiteRT for Windows and Android devices.
• 😀 A major feature of WebNN is its ability to choose the best hardware device (CPU, GPU, or NPU) for model execution, optimizing performance based on available resources.

Q & A

What is WebNN and why is it being developed?

WebNN is an upcoming web standard API for executing neural network graphs. It is being developed to provide a way for web developers to leverage the power of AI accelerators like CPUs, GPUs, and NPUs (Neural Processing Units) without sacrificing the benefits of the web platform, such as its low friction and linkability.

What role do CPUs, GPUs, and NPUs play in WebNN?

CPUs are great for universal reach and smaller workloads, GPUs provide more power and efficiency for intensive tasks, and NPUs (or AI accelerators) offer even more specialized efficiency, especially for long-running tasks, by using dedicated compute resources. WebNN aims to allow the web to access all of these hardware resources seamlessly.

What is the main challenge WebNN is addressing for developers?

WebNN aims to solve the problem of developers having to choose between the performance of native platforms and the convenience of the web. It allows web applications to access high-performance AI computing resources, such as NPUs, without sacrificing the web's key features.

How does WebNN integrate with different hardware platforms?

WebNN integrates with hardware platforms like CPUs, GPUs, and NPUs by providing a unified API that interacts with the hardware-specific drivers. It uses various execution providers, such as ONNX Runtime and LiteRT, to optimize neural network execution on different platforms.

What are the different execution providers that WebNN uses?

WebNN uses several execution providers depending on the platform: ONNX Runtime (open-source) for various devices, LiteRT for CPU acceleration on Windows and Google platforms, Core ML on Apple devices for accessing Apple's neural engine, and DirectML on Windows for integrating with hardware vendors like Intel, NVIDIA, AMD, and Qualcomm.

What is an origin trial in the context of WebNN?

An origin trial is a way for developers to try out new web features, like WebNN, on their websites before they are fully released. It allows them to give feedback and helps improve the API before it becomes a standard.

How does WebNN improve AI processing in web browsers?

WebNN enhances AI processing in web browsers by providing a standard way to execute machine learning models directly in the browser using the available hardware resources (CPU, GPU, or NPU). This allows web developers to implement AI features in their applications with optimized performance.

How does WebNN interact with WebGPU for better performance?

WebNN can work in conjunction with WebGPU to avoid unnecessary round trips between the JavaScript process and the GPU. This improves performance by allowing inference results to stay on the GPU and be directly displayed without returning to the JS process.

What are the next steps for the WebNN API development?

The next steps include expanding support for more machine learning frameworks, improving adapter selection for device preferences, ensuring seamless interoperability with WebGPU, and adding more custom operations for advanced use cases.

How can developers start using WebNN in their projects?

Developers can start using WebNN by integrating it into their projects via ONNX Runtime for the web, or by using other supported frameworks. They can choose their execution provider (e.g., CPU, WebGPU, or WebNN) and start building machine learning inference workflows in their applications.