AI Doesn’t Win Games - Decisions Do | with Mike Band (NFL) | AI in Decision Making

Summary

In this episode of Digital Storm Weekly, Mike Bent, Senior Manager of Research and Analytics at NFL NextGen Stats, dives deep into how AI and data analytics are transforming the NFL. He highlights the importance of using AI outputs as tools while still relying on expert intuition, and explores how machine learning models, including deep learning and transformers, are shaping player tracking and game strategies. Mike also discusses challenges like latency in real-time data processing and the future of personalized fan experiences, urging business leaders to rethink AI not just as a tech problem, but a crucial business asset.

Takeaways

• 😀 Sports analytics provides rich, controlled data that can be applied to other industries, demonstrating the value of thinking outside the box with data-driven decisions.
• 😀 AI models are only as good as the context they have access to; decision makers should still trust their gut, as their experience provides crucial context that AI may lack.
• 😀 Data preparation and processing account for 80% of the analytics pipeline; the model itself is only 20%, emphasizing the importance of clean and well-structured data.
• 😀 Latency is a major challenge in sports analytics, particularly when trying to provide real-time or near real-time stats during broadcasts with high accuracy.
• 😀 The NFL’s use of AI has advanced from basic tracking data and simple models to complex neural networks and transformer architectures for enhanced accuracy in player performance analysis.
• 😀 In sports, certain data insights, like success rate, may be overlooked or misunderstood by fans but can provide valuable information about team performance over time.
• 😀 Despite advancements in AI and machine learning, human experts still play a vital role in interpreting and validating AI outputs, particularly when errors or hallucinations occur.
• 😀 The introduction of transformer architecture has significantly boosted the accuracy of coverage models in football, improving defensive analysis.
• 😀 5 years ago, certain metrics were difficult to calculate due to limitations in machine learning models. Now, advances in AI, like transformers, enable deeper insights.
• 😀 AI-driven decision-making in sports is a controlled environment with factual data, but it is still important for AI to be tested for its accuracy and reliability in corporate settings.
• 😀 Corporate leaders can implement AI in areas like reporting, where AI tools can assist in generating insights and scaling up research efforts, similar to how NFL teams scale analytics.

Q & A

Why should a CEO, CIO, or CTO listen to this episode?

Because the NFL provides a controlled environment with rich data that illustrates how analytics and AI can be applied to complex decision-making. Leaders can learn to apply similar approaches to their organizations to make data-informed decisions.

What is the most overrated football statistic according to Mike Bent?

Passer rating is considered the most overrated football stat, as it does not fully capture player or team performance.

What skill is essential for future NFL analysts?

Speaking the language of football is essential, as it enables analysts to interpret data correctly and communicate insights effectively.

How does AI complement human decision-making in the NFL?

AI provides data-driven insights, but human expertise and gut instincts remain critical for interpreting outputs, spotting errors, and making nuanced decisions.

What part of the analytics pipeline is more important than the model itself?

Data preparation and preprocessing are more important, accounting for about 80% of the effort, whereas the model itself accounts for only 20%.

What is one NFL moment where the data told a different story than fans expected?

During the AFC Championship, the Denver Broncos went for it on fourth down, which was statistically correct, even though fans expected them to take a field goal. The outcome differed from expectations, but the process was sound.

What is a key challenge of using AI and machine learning with tracking data?

The biggest challenge is latency — ensuring that tracking data and event labeling are accurate down to the tenth of a second so that machine learning models provide reliable outputs in real time.

How has transformer architecture improved NFL analytics?

Transformers have enabled more accurate defensive coverage models, identifying which player was responsible for which receiver and improving labeling accuracy compared to older neural networks.

What lessons can corporate leaders learn from NFL AI practices?

Corporate leaders can learn to scale repetitive tasks like reporting using AI, validate AI outputs against facts, balance real-time insights with accuracy, and focus on high-quality data and domain expertise.

What future experiences will NFL fans take for granted in 5 years?

Fans will have highly customized and personalized viewing experiences, including alternative broadcasts, analytics-focused content, and tailored content before, during, and after games.

How does the NFL balance real-time accuracy versus broadcast latency?

They provide directionally accurate stats immediately for live broadcasts, then finalize precise values after QA. This ensures fans receive timely insights without compromising long-term accuracy.

Why is gut instinct still valuable in analytics-heavy environments?

Because human experience provides contextual understanding that AI may not fully capture. Subject-matter experts can identify errors, hallucinatory outputs, or missing context in AI recommendations.