5. November 2024

Summary

In this keynote, the speaker outlines the key advantages of VMware's Private AI platform, focusing on privacy, agility, and resource management. VMware's AI solution provides businesses with faster setup, superior performance, and up to 3x lower costs compared to public AI services. Key features include a model store for managing AI models, centralized operations, GPU visibility for efficient workload placement, and data indexing for AI model training. Additionally, VMware integrates Azure AI Video Indexer for localized training. The platform enables flexible, scalable AI services with enhanced TCO and simplified operations for enterprises.

Takeaways

• 😀 Privacy is a top priority when deploying AI solutions, ensuring data protection while processing AI models.
• 🚀 VMware’s Private AI platform offers rapid AI deployment, standing up AI services in minutes rather than weeks or months.
• ⚙️ VMware’s distributed resource scheduler optimizes resource management, effectively provisioning GPUs, memory, network capacity, and data for AI workloads.
• 💸 Businesses can save up to 3x in costs by using VMware’s Private AI platform compared to public AI services.
• 🎯 VMware’s platform provides equal or superior performance to bare-metal infrastructure, thanks to advanced scheduling technology developed over two decades.
• 🛠️ The platform offers flexibility by supporting a wide range of hardware, OEMs, system integrators, tools, and models for AI applications.
• 🔐 The new Model Store feature helps govern AI models, enabling scanning, approval, and management before use in production environments.
• 💻 GPU profiling and visibility tools provide insights into resource usage, optimizing workload placement and performance across GPUs and memory.
• 📊 Data indexing and retrieval services enable efficient preparation and integration of private data into AI models for better training outcomes.
• 🗣️ The Agent Builder tool allows users to create AI applications using natural language, simplifying the development process.
• 🌐 VMware integrates with public cloud services like Azure AI, running AI applications like the video indexer on private infrastructure for better resource utilization.

Q & A

Why are organizations choosing to partner with Broadcom and VMware for private AI?

Organizations are partnering with Broadcom and VMware for private AI due to key factors like privacy, speed, agility, and cost-effectiveness. VMware provides an AI platform that ensures data control, fast AI service setup, scalability, and resource management, all while offering up to 3x lower total cost of ownership compared to public AI platforms.

What is the main advantage of VMware’s AI platform compared to traditional AI service deployments?

The main advantage of VMware's AI platform is its ability to stand up AI workstations and services in just minutes, compared to the weeks or months it typically takes with traditional setups. This speed and agility give organizations flexibility in adapting to new AI models as they emerge.

How does VMware ensure superior performance in virtualized AI environments?

VMware ensures superior performance through its sophisticated resource scheduler, which optimizes workload placement across not just GPUs, but also memory, network capacity, and data. This scheduler is the result of two decades of development, enabling efficient resource management and minimizing performance sacrifice in virtualized environments.

What is the significance of VMware’s distributed resource scheduler for AI workloads?

The distributed resource scheduler is essential for effectively managing AI workloads. It ensures that AI applications are allocated the right amount of network, memory, and data resources, alongside GPUs, enabling optimal performance and preventing resource bottlenecks during execution.

What cost savings can organizations expect when using VMware’s private AI platform?

Organizations can expect cost savings of up to 3x lower than comparable public AI services. This significant reduction in cost is achieved through efficient resource management, centralized operations, and the flexibility of VMware's private AI platform.

How does VMware’s private AI platform maintain a competitive edge in terms of performance?

VMware’s private AI platform maintains a competitive edge in performance through its advanced scheduling capabilities. By leveraging years of experience running multiple operating systems and applications simultaneously, VMware’s platform optimizes resources and achieves performance comparable to or exceeding bare metal setups.

What new features were introduced in VMware Cloud Foundation 9 for AI applications?

VMware Cloud Foundation 9 introduces several new features, including the Model Store for managing AI models, guided deployment tools for easy AI stack installation, GPU visibility for better workload placement, data indexing for preparing private data for AI models, and an agent builder for creating custom AI services using natural language.

What is the Model Store and how does it help with managing AI models?

The Model Store, introduced in VCF 521, allows users to govern and manage AI models effectively. It integrates with popular container registries like Harbor and supports scanning, approval, and management of models, ensuring they meet security and operational standards before being deployed.

How does the agent builder feature enhance user experience in AI services?

The agent builder allows users to interact with AI services using natural language, making it easier to build and customize AI-driven solutions like support chatbots without requiring extensive programming knowledge. This improves accessibility and user experience.

Can VMware's platform support public AI services running on private infrastructure?

Yes, VMware's platform supports the integration of public AI services running on private infrastructure. An example is the Azure AI Video Indexer, which can run on VMware Cloud Foundation, enabling organizations to use public AI services on their private servers, making it easier to scale and manage AI workloads.