Amazon’s AI Roadmap With AWS CEO Garman

Summary

In an insightful conversation, CEO Matt Garman reflects on his first year at the helm, highlighting AWS's rapid AI advancements, particularly in cloud migration and generative AI. He emphasizes the growing demand for AI inference, the importance of customer choice in platforms, and the collaboration with partners like Anthropic. Garman also touches on AWS's global expansion, including new regions in Latin America and Europe, and discusses innovations in AI infrastructure. He stresses that AI is set to revolutionize industries and work processes, while AWS continues to provide cutting-edge, scalable solutions for customers worldwide.

Takeaways

• 😀 The CEO reflects on a successful first year in the role, marked by rapid technological innovation and customer adoption of new AI technologies.
• 😀 Cloud migration has accelerated significantly, with more customers moving entire estates into the cloud, especially in the context of AI and generative technologies.
• 😀 AWS infrastructure plays a central role in supporting customers' AI models, including first-party models like Amazon Nova and third-party models like Anthropic.
• 😀 The AI business is at a multi-billion dollar run rate, with a focus on the early stages of AI’s impact on every industry, job, and business.
• 😀 Amazon uses generative AI across various platforms, such as optimizing fulfillment centers, summarizing product reviews, improving Alexa functionality, and automating software development through Amazon Queue.
• 😀 The majority of AI workloads today are focused on inference rather than training, with inference expected to dominate future usage as AI becomes embedded in all applications.
• 😀 Inference is becoming a core building block in every application, similar to how compute, storage, and databases are fundamental to modern systems.
• 😀 AI-related revenue continues to grow, with specific reference to increasing use of AI across different sectors and Amazon's continuous push for innovation in the field.
• 😀 AWS is working closely with Anthropic on Project Right, building one of the largest compute clusters to train next-gen AI models, with significant improvements in performance and cost-efficiency.
• 😀 There’s strong demand for both training and inference technologies, with AWS offering a diverse range of solutions, including custom-built processors and NVIDIA technologies, while ensuring that customers have choices and flexibility in their AI workloads.
• 😀 AWS continues to expand its global data center presence, including new regions in Mexico, Chile, Brazil, and Europe, with a special focus on the European Sovereign Cloud to address data sovereignty concerns for EU-regulated workloads.

Q & A

What has been the biggest achievement in the CEO's first year at the company?

The biggest achievement has been the rapid pace of innovation, especially how quickly customers are adopting new technologies, including AI and cloud migration. The CEO highlights the increasing trend of customers moving their entire estates into the cloud and the exciting development of AI technologies.

What is the significance of Amazon's AI business hitting a multi-billion dollar run rate?

The significance lies in the fact that this represents the early stages of AI transformation across industries. Although it's currently at a multi-billion dollar run rate, it is just the beginning, and the CEO believes AI will transform every business, industry, and job in the future.

What proportion of Amazon's AI revenue comes from AWS infrastructure?

The CEO confirms that AWS accounts for the majority of the AI revenue, with customers using AWS to run their models. The revenue also comes from Amazon’s own hosted models and applications like Amazon Queue, as well as customers building on top of AWS infrastructure.

How does the CEO view the future of AI in terms of usage for training and inference?

Currently, inference is the dominant use case, with more AI workloads shifting towards inference as models become larger and more usage is generated. The CEO anticipates that the vast majority of AI usage will eventually be in inference, as it becomes embedded into all applications.

What is the current distribution between AI training and inference workloads?

Inference now outpaces training in terms of usage. In the past, AI usage was mostly focused on training large models with small usage, but as the models grow and demand for inference increases, this shift is happening quickly.

Why does the CEO believe tokens are not the best measure of AI's work?

The CEO explains that while tokens are useful for measuring text generation, they don't capture the full scope of AI's work. Some models can process and reason over long periods before producing output, and this process is not fully reflected in token output alone, especially in tasks like coding or image generation.

What is Project Right and its significance?

Project Right is a collaboration with Anthropic to build one of the largest compute clusters for AI model training. This cluster will be used to train Anthropic's next generation of AI models, including the launch of Cloud 4. The CEO highlights the scale and performance of the project as crucial to AI advancements.

What are the challenges and innovations Amazon is focusing on to reduce the cost of AI?

The main challenges include the high costs of AI today. To reduce costs, Amazon is focusing on innovations in both silicon (hardware) and software (algorithmic improvements). This involves using custom-designed processors like Uranium two to deliver cost-effective performance at scale.

What is the competition between Amazon and Nvidia in AI?

While both Amazon and Nvidia play significant roles in the AI space, the CEO emphasizes that there is room for both platforms. Amazon collaborates with Nvidia and uses their latest technology, but also continues to push its own innovations, such as its custom-designed processors like Tidjane Thiam two.

How does Amazon view the use of Anthropic's AI models on other platforms like Azure?

Amazon is open to Anthropic's models being available on other platforms like Azure. The CEO emphasizes that Amazon's goal is to make AWS the best place for AI workloads, but acknowledges that customers want choice and flexibility in where they run their AI applications.