What is the ideal team structure for machine learning? - Question of the Week

Summary

The speaker discusses challenges and potential improvements in machine learning team structures within organizations. Current models often separate machine learning teams as specialized subsystems, leading to bottlenecks and slow feedback loops. To address these issues, the speaker suggests a shift towards a more integrated approach, where machine learning expertise is embedded within stream-aligned teams, supported by platform services that enable self-sufficiency. This model promotes better alignment with business outcomes by encouraging teams to focus on real-world results, not just creating models, fostering faster decision-making and more effective use of machine learning.

Takeaways

• 😀 Traditional machine learning (ML) teams are often seen as complex subsystems, responsible for creating and training models, but they face several challenges in this structure.
• 😀 A common problem with the traditional model is the creation of bottlenecks, as ML teams may not have enough capacity to handle multiple requests for new or evolving models.
• 😀 Slow feedback loops are a significant issue with the traditional approach, as teams that run the models may need to send feedback back to the ML team, leading to delays in improvements.
• 😀 ML teams often lack exposure to modern software engineering practices (e.g., version control, continuous integration, testing), which can hinder their efficiency and the long-term maintenance of models.
• 😀 To improve sustainability, ML teams should adopt modern platforms that can scale and deliver models more efficiently, along with standardized methods for deploying them.
• 😀 An enabling team focused on modern software engineering practices should help ML teams adopt best practices in versioning, testing, and optimization to enhance their performance.
• 😀 ML teams should include strong product management capabilities to prioritize model requests, understand business needs, and support models over time, ensuring they align with overall business goals.
• 😀 A more evolved model suggests integrating ML capabilities into stream-aligned teams, where these teams would be able to consume ML platform services for self-sufficient model creation.
• 😀 Platform services should abstract complex tasks, such as data preparation, model training, and testing, to allow stream-aligned teams to leverage ML without deep expertise in the field.
• 😀 Stream-aligned teams, instead of being incentivized to create more complex models, should focus on achieving business outcomes and making data-driven decisions on when to use ML models.
• 😀 The new approach should enable quicker decision-making by stream-aligned teams, who can switch to simpler solutions (e.g., non-ML alternatives) if they deliver better business results, aligning incentives with business goals.

Q & A

What is the traditional role of machine learning teams in most organizations?

In most organizations, machine learning teams are considered 'complicated subsystems' responsible for model creation, training, and deployment. Once the model is ready, it is handed over to another team, often operations or stream-aligned, for production use.

What challenges arise from the traditional machine learning team model?

Challenges include slow feedback loops, potential bottlenecks due to limited capacity, and the machine learning teams lacking exposure to modern software engineering practices like version control, testing, and performance monitoring.

What would make the current machine learning team model more sustainable?

For sustainability, the model would need modern platforms that scale deployments, enabling teams to help ML teams adopt best practices, and strong product management within the teams to ensure alignment with business objectives and continuous support of model evolution.

What role does an enabling team play in improving machine learning teams?

An enabling team helps ML teams adopt modern software engineering practices such as version control, continuous integration, automated testing, and performance improvement, effectively boosting their efficiency and performance.

Why is product management critical within machine learning teams?

Product management within ML teams ensures that models are aligned with business needs. It also helps prioritize and support models effectively over time, ensuring continuous improvement and alignment with evolving business goals.

What is the alternative model to the 'complicated subsystem' approach for machine learning teams?

An alternative model decentralizes machine learning capabilities, allowing stream-aligned teams to use platform services that abstract away the complexity of ML. These teams can develop and manage models more autonomously, focusing on achieving business outcomes.

How does the new model improve business decision-making in ML?

By focusing on business outcomes rather than the creation of models themselves, stream-aligned teams can more effectively decide when to use complex models or simpler solutions based on real-time system feedback, thus improving decision-making and efficiency.

What role do platform services play in the new machine learning model?

Platform services abstract the complexities of machine learning, providing tools for data preparation, model training, and testing. These services enable stream-aligned teams to use ML capabilities without deep expertise, making it easier to integrate ML into business processes.

How do stream-aligned teams benefit from the new model in terms of autonomy?

Stream-aligned teams benefit by becoming more self-sufficient, as they can leverage platform services to create models and make decisions independently, without relying heavily on centralized ML teams. This increases agility and responsiveness.

Why is the shift from a complicated subsystem to a stream-aligned model potentially more effective for businesses?

This shift helps ensure that machine learning efforts are directly tied to business value. It allows teams to quickly adapt models based on feedback and business needs, optimizing both model efficiency and overall business outcomes.