Rising Titan – Mistral's Roadmap in Generative AI | Slush 2023

Summary

In a recent discussion at Slush, Paul Murphy, a partner at Light Speed, interviews Arthur, the co-founder of Mistral, a company focused on developing foundational AI models. Arthur explains that Mistral's vision is to create state-of-the-art models that are accessible to developers, allowing them to specialize and customize models for their specific tasks. They have already released a 7B model and have seen significant community engagement, with developers creating derivative works and integrating the model into various open-source projects. The conversation also touches on the importance of regulation in AI, with a focus on product safety and the need for empirical evidence in discussions about national security and existential risks. Arthur emphasizes the potential of AI to revolutionize sectors like healthcare and education and believes that fostering an open AI community is crucial for addressing global challenges like climate change. He stresses the significance of having strong European actors in AI to drive technological advancements and policy proposals that align with European values.

Takeaways

• 🤖 **Investment in AI**: Light Speed, a Silicon Valley based fund, has been investing in Europe since 2007 with a focus on AI, investing over a billion dollars into the category.
• 🚀 **Mistral's Vision**: Mistral, a startup co-founded by Arthur, aims to develop state-of-the-art AI models quickly and provide open access to developers for specialization.
• 🧠 **Open Source Models**: Mistral's approach is to create easy-to-use open source models that enable developers to tailor large language models for their specific applications.
• 💾 **Data and Training**: The success of Mistral's 7B model was attributed to a good team, a machine learning Ops system, and a focus on creating high-quality datasets.
• 🔍 **Community Engagement**: The release of the 7B model led to thousands of derivative works, showcasing new capabilities and integrations within the open source community.
• 🌐 **Upcoming Developments**: Mistral has plans for new models, techniques, and a platform offering hosting capacities with fast inference capabilities to be announced by year-end.
• 🧑‍💼 **Building a Company**: Arthur highlights hiring the best talent and community engagement as key challenges while building Mistral.
• 📈 **Regulation and Safety**: There's a call for hard regulation on the product side for safety and compliance, with a focus on empirical evidence over speculation.
• 🌱 **Open Science**: Open source principles have accelerated AI advancements, and Mistral aims to continue this tradition by fostering knowledge circulation.
• 🛡️ **Product Safety**: Arthur suggests that regulation should focus on the application layer, ensuring that deployed applications meet safety standards.
• ⚖️ **Independence in Regulation**: For effective regulation, there's a need for independent oversight, possibly state-funded, to prevent industry bias and pressure.
• 🌟 **Positive Impact**: AI has the potential to revolutionize sectors like healthcare and education, enabling more efficient and personalized services.
• 🌍 **European Leadership**: It's important for Europe to have strong AI actors to drive technological advancements and influence policy to reflect European values.

Q & A

What is the name of the company that Paul Murphy is a partner at?

Paul Murphy is a partner at Light Speed, a Silicon Valley based fund with investments in Europe since 2007.

How long has Light Speed been investing in Europe?

Light Speed has been investing in Europe since 2007.

What is the focus of the company Mistral, as described by Arthur?

Mistral focuses on developing state-of-the-art models quickly and aims to provide open access to these models for developers to specialize and make them their own, thus creating more human-like intelligent applications.

What was the first major milestone for Mistral after securing their seed round of funding?

The first major milestone for Mistral was to build their 7B model, which they achieved in less than three months.

How did Mistral manage to develop their 7B model so quickly?

Mistral managed to develop their 7B model quickly by having a good team, creating a machine learning Ops system, focusing on good training and inference code bases, and dedicating a large part of the team to curate and optimize datasets.

What has been the community's response to Mistral's 7B model?

The community has been very engaged with the 7B model, with thousands of derivative works where developers fine-tuned the model for their specific tasks or datasets, resulting in new capabilities and applications.

What are the next steps for Mistral?

Mistral is working on new models, techniques, and the beginning of a platform. They plan to offer hosting capacities for their models with fast inference capabilities and are expected to announce these developments before the end of the year.

What are the two main challenges that Arthur identifies for building the company?

The two main challenges Arthur identifies are hiring the best engineers and scientists in a competitive landscape and creating an engaged community around their open-source models.

How does Arthur view the concept of open source in the context of AI models?

Arthur differentiates between open source software and open weight models in AI. While providing the weights allows for modification, it doesn't necessarily enable full understanding due to the opacity of the models. He also mentions a balanced approach between openness and maintaining a competitive edge.

What is Arthur's perspective on the role of regulation in AI?

Arthur believes that regulation should focus on product safety, ensuring that AI applications meet certain safety standards. He also emphasizes the importance of empirical evidence in discussions around national security and existential risks related to AI.

How does Arthur envision AI improving society in the future?

Arthur sees AI as a tool that can revolutionize healthcare, education, and enable more creative thinking in society. He also believes AI can contribute to addressing existential risks like climate change by unlocking new scientific discoveries.

Why is it important for Europe to have a strong presence in the field of AI, according to Arthur?

Arthur believes it's crucial for Europe to have strong technological actors in AI to drive the field forward, shape the technology according to European values, and ensure Europe is not just a spectator as the technology transforms society.