DevOps Huddle EP 19 | Measuring GitHub Copilot's Downstream Impact with DORA | Opsera

Summary

In this episode of the DevOps Huddle, the discussion focuses on measuring the impact of GitHub Copilot and its effect on downstream processes using DORA metrics. The webinar explores the challenges of aggregating data across tools and the importance of these metrics for understanding software delivery performance. It introduces a solution that connects GitHub Copilot usage with DORA metrics, offering insights into developer productivity and efficiency. The conversation also includes a 14-day free trial offer for a dashboard that integrates GitHub Copilot and DORA metrics.

Takeaways

• 😀 The webinar is part of a three-part series focusing on unified insights capabilities, particularly the impact of using GitHub Copilot and how it can be measured with Dora.
• 🔧 Part one of the series covered GitHub Copilot's developer activity, actual usage, and licensing, while part two discusses the downstream impact after code commits.
• 👋 Introductions of the panelists, Ed, a sales engineer with a background in devops and Gilbert, VP of Post Services with experience in building devops teams.
• 📊 A poll was conducted to gauge the audience's use of GitHub Copilot and GitHub, showing an even split between those who are and aren't using GitHub Copilot.
• 📅 An upcoming third part of the series is announced for July 25th, focusing on uniting GitHub Copilot with developer experience and security posture.
• 🔬 Dora is introduced as a 10-year research and assessment program run by Google, aimed at understanding capabilities and processes that drive higher delivery performance.
• 📈 Dora focuses on four core metrics: Lead Time for Change, Deployment Frequency, Change Failure Rate, and Mean Time to Resolution (MTTR).
• 🛠 Ed explains the importance of not creating your own metrics but leveraging the established Dora metrics for measuring software delivery performance.
• 🔀 Gilbert discusses the challenges of setting up Dora metrics, including data aggregation across tools, time pressure, risk management, and updates/maintenance.
• 🔄 The webinar highlights the importance of distinguishing between inner loop (developer activity) and outer loop (system metrics) when considering metrics for devops and development teams.
• 🔗 The final part of the webinar demonstrates how GitHub Copilot usage can be associated with Dora metrics to show the impact on organizational software delivery performance.

Q & A

What is the main focus of the 'devops Huddle, episode 19' webinar?

The webinar focuses on the unified insights capabilities, particularly the new GitHub Copilot measuring capabilities, and its downstream impact on software development processes.

What is GitHub Copilot and what does it aim to improve?

GitHub Copilot is an AI programming assistance tool that aims to improve developer productivity and efficiency by providing code suggestions and automating certain coding tasks.

What does part one of the webinar series cover?

Part one of the series is about understanding GitHub Copilot, measuring developer activity, actual usage, and licensing to determine how much of the license is being utilized.

Who are the panelists introduced in the webinar, and what are their backgrounds?

The panelists are Ed, a sales engineer at Opsera who has a background in development and devops, and Gilbert, VP of Post Services at Opsera, who has experience in building devops teams and processes.

What is Dora, and what does it stand for?

Dora stands for the DevOps Research and Assessment program, a 10-year-long research initiative run by Google to understand what capabilities, technologies, and processes drive higher delivery performance in software development.

What are the 'Dora Core 4' metrics that Ed discusses in the webinar?

The 'Dora Core 4' metrics are lead time for change, deployment frequency, change failure rate, and mean time to resolution (MTTR), which are key performance indicators for measuring software delivery and organizational performance.

Why is aggregating data across different tools a challenge when implementing Dora metrics?

Aggregating data is challenging because it involves collecting data from various tools used across different teams, which may have different combinations of tools and processes, and then ensuring the data is consistent and valid for accurate Dora metric calculations.

What is the significance of the 14-day free trial mentioned in the webinar?

The 14-day free trial allows participants to connect their existing tools to the Opsera platform, get started with measuring GitHub Copilot usage and Dora metrics, and evaluate the benefits without any initial commitment.

How can GitHub Copilot's impact on an organization be measured?

The impact of GitHub Copilot can be measured by associating its usage data with Dora metrics, which provide insights into software delivery performance and help demonstrate the return on investment for using GitHub Copilot.

What is the purpose of the third part of the webinar series scheduled for July 25th?

The third part of the series will focus on uniting GitHub Copilot with developer experience and ensuring that the security posture of the organization remains safe, while also exploring the satisfaction of developers with their work and the impact on business security.

What are the inner loop and outer loop in the context of software development metrics?

The inner loop refers to developer-centric metrics focusing on activity and efficiency, such as coding and meeting productivity. The outer loop, or Dora metrics, refers to system productivity metrics that measure the performance of the entire software delivery process, such as lead time for changes and deployment frequency.