Bell's data modernization journey: executing for the future with Google

Summary

Eric from Bell's Enterprise data platform team shares his journey from intern to architect and discusses the company's transition to Google Cloud. He highlights three key mistakes made during their on-premise data management: proliferating data silos, neglecting developer productivity, and poor data governance practices. Eric outlines how Bell is addressing these issues by standardizing on a single data platform, improving developer tools, and adopting data cataloging with Google Cloud's Dataplex.

Takeaways

• 📊 Bell is Canada's largest telecom provider, serving multiple provinces and regions.
• 🕹️ The speaker, Eric, works in Bell's Enterprise Data Platform team and has been with the company for 17 years, starting as an intern.
• 🏆 The speaker used his experience in competitive gaming to draw parallels between analyzing mistakes in games and reviewing past data management strategies.
• ☁️ Bell has recently migrated to Google Cloud and is reviewing past mistakes to avoid repeating them in the new environment.
• 🏗️ The first major mistake identified was the creation of numerous data silos, leading to inefficiencies and challenges in data management.
• 🚧 The second mistake was the lack of focus on improving developer productivity, leading to redundant and inefficient coding practices.
• 🛠️ Bell is now working on standardizing their data platform, consolidating their data into Google BigQuery, and improving data accessibility.
• 💻 The company is also implementing tools like Terraform modules and Backstage to standardize and streamline developer workflows.
• 🔄 Bell plans to create config-driven libraries to reduce redundant coding and improve the efficiency of data pipeline creation.
• 📑 The third mistake involved poor data governance practices, with metadata managed in Excel and Confluence. Bell is now standardizing on DataPlex for better data cataloging and governance.

Q & A

Who is the speaker, and what is his role at Bell?

The speaker is Eric, who works in the Enterprise Data Platform team at Bell. He has been with the company for 17 years, starting as an intern and then moving into roles such as Data Engineer and currently, an Architect.

What is Bell, and what regions does it serve?

Bell is the largest telecommunications provider in Canada, serving regions like Quebec, Ontario, Atlantic provinces, and Manitoba.

Why does the speaker mention playing competitive video games, and how does it relate to the talk?

The speaker mentions playing competitive video games like Rocket League to draw a parallel between analyzing mistakes in gaming and analyzing mistakes made during Bell's data platform migration. The idea is that reviewing mistakes is crucial for improvement in both contexts.

What were the three major mistakes Bell made with their on-premise data systems?

The three major mistakes were: 1) Deploying dozens of data environments and creating data silos. 2) Failing to improve developers' productivity and experience. 3) Poor data governance practices, including the management of metadata in Excel spreadsheets and Confluence.

What was the impact of deploying multiple data environments at Bell?

Deploying multiple data environments led to a complex web of data pipelines, making it difficult for engineers to create value and requiring them to spend a lot of time moving data around instead of solving core problems.

How did Bell plan to address the issue of multiple data environments in their migration to Google Cloud?

Bell planned to standardize on a single data platform in Google Cloud, specifically using Google BigQuery. This would centralize data and reduce the need for engineers to move data across different environments.

What problems did Bell encounter with developer productivity, and how are they addressing these in the cloud migration?

Bell found that developers were repeatedly solving the same problems and writing redundant code, particularly in areas like data compaction. To address this, they are building a solid foundation with Terraform modules and using tools like Backstage to standardize and automate processes, improving productivity.

What is Backstage, and how is Bell utilizing it in their cloud migration?

Backstage is an open-source developer portal created by Spotify that allows engineers to use templates to quickly spin up applications. Bell is using Backstage to simplify and standardize their development processes during their cloud migration.

What issue did Bell face with data governance, and what tool are they using to improve it?

Bell struggled with managing table metadata using Excel spreadsheets and Confluence, leading to inefficiencies. To improve data governance, they are now standardizing with Google DataPlex to catalog assets and provide a centralized location for data governance.

What is the speaker's main message to the audience regarding their own experiences with large-scale migrations?

The speaker encourages the audience to learn from Bell's mistakes and shares their experiences to help others avoid similar pitfalls. He invites others with experience in large-scale migrations to reach out and share their learnings, particularly as Bell prepares for a significant migration next month.