Choosing the right gen AI partner

Summary

In this discussion, leaders from Vellum AI and Rev share insights on building AI-driven companies, focusing on generative AI and platform integration. Akash Sharma of Vellum AI and Tyung Park of Rev highlight the importance of factors like scalability, security, and model performance when selecting AI partners. They emphasize the role of foundational models, the need for effective cloud infrastructure, and balancing development speed with reliability. The conversation also covers strategies for AI deployment, maintenance, and the importance of a supportive ecosystem for startups to achieve success in the evolving AI landscape.

Takeaways

• 😀 Akash Sharma is the CEO and founder of Vellum AI, which is an end-to-end LLM Ops platform supporting generative AI use cases for developers.
• 🤖 Vellum AI uses Google's Vertex AI due to its ease of use and wide range of models, balancing quality, cost, latency, and privacy.
• 🖥️ Tyung Park is the Chief Scientist at Rev, a company focused on providing more control for visual content creators through non-text-based visual content generation.
• 💡 Rev needed thousands of GPUs for training a new visual foundational model and prioritized factors like reliability, GPU interconnect, ecosystem, and availability when selecting a cloud provider.
• 🔐 Akash emphasized the importance of early compliance with security and privacy standards, like SOC 2, HIPAA, and GDPR, especially for enterprises working with sensitive data.
• ⚙️ Tyung highlighted the critical role of GPU interconnect speed and reliability, noting that fast communication between GPUs significantly improved development cycles.
• 📊 Both speakers stressed the need for test-driven development and continuous monitoring when building generative AI applications, with well-defined test cases and metrics.
• 🚀 Akash recommended integrating generative AI solutions into existing systems using a platform that provides both a UI and IDE, allowing cross-functional teams to collaborate efficiently.
• 🌐 Tyung found Google's unique offering of foundational models and cloud platform helpful for scaling AI solutions, allowing for easier access to compute resources and model development.
• 🤝 Both speakers highlighted the importance of human support and strategic partnerships alongside the technical infrastructure for startups to succeed with generative AI solutions.

Q & A

What are the companies being represented in the discussion?

The companies represented are Vellum AI, founded by Akash Sharma, and Rev, where Taung Park serves as the Chief Scientist.

What is Vellum AI's primary focus?

Vellum AI is an end-to-end LLM Ops platform that supports generative AI use cases for developers, offering tools for prompt engineering, evaluations, document retrieval, and workflow orchestration.

Why did Vellum AI choose Google Cloud as their AI provider?

Vellum AI initially started with another cloud provider but found that Google Cloud, specifically Vertex AI, provided better abstractions and ease of use. They also appreciated the flexibility and options of Google's Gemini models.

What challenge is Rev aiming to solve with generative AI?

Rev aims to enable controllable visual content generation, allowing users to manipulate elements within generated images, such as resizing or spatial rearrangement, which text-based input tools struggle to provide.

What were the key criteria Rev used to select their AI provider?

Rev's selection criteria included reliability, fast interconnect between GPUs, a supportive ecosystem for GPUs, and quick availability of the GPUs they needed.

Why is security and privacy a critical consideration for Vellum AI?

Security and privacy are critical for Vellum AI as they cater to enterprises that often require compliance with frameworks like SOC 2, HIPAA, ISO, and GDPR. Ensuring data privacy is especially important when working with heavily regulated industries like finance and healthcare.

How does Rev manage the scalability of its generative AI solution?

Rev emphasizes GPU interconnect speeds and reliability to ensure their systems scale efficiently. They have found Google Cloud's A3 high and A3 mega instances helpful due to their fast inter-GPU connectivity, which minimizes bottlenecks.

What is a test-driven development approach in AI, as described by Akash Sharma?

A test-driven development approach involves defining test cases and evaluation metrics upfront, iterating on the AI model until it meets quality, cost, and latency criteria, deploying in a real-world environment, and continuously monitoring and updating based on new data.

How does Rev leverage foundational models like Google's Gemini?

Rev uses foundational models like Gemini to assist in tasks such as data annotation, where they can ask the model to detect and filter out inappropriate content like violent images, which would otherwise require training a specialized classifier.

Why is human support important alongside AI technology according to the discussion?

Human support is crucial because AI development requires not just technical capabilities but also strategic guidance, customer support, and collaboration with engineers to ensure the smooth implementation and scaling of AI solutions.