Managing Cymbal Superstore’s cloud solutions

Summary

The video explains the role of an Associate Cloud Engineer at Cymbal Superstore after deploying and implementing their cloud solutions. It highlights managing Google Cloud resources, monitoring with Cloud Observability, and configuring services like Compute Engine, GKE, and Cloud Functions. The focus shifts to maintaining successful operations of cloud resources, such as updating instance templates, setting up load balancers, and querying data from Bigtable using BigQuery. Engineers are also tasked with providing pipeline data information and managing data streams, ensuring smooth functionality of cloud applications like supply chain, ecommerce, and transportation management systems.

Takeaways

• 🔧 The role of an Associate Cloud Engineer transitions to maintaining successful cloud operations after deployment.
• 🌐 An Associate Cloud Engineer needs to manage compute, storage, and networking resources on Google Cloud.
• 👀 Monitoring and logging using Google Cloud Observability are key skills for ensuring successful operations.
• 🏙️ Cymbal Superstore’s supply chain management app is hosted near their headquarters in Minneapolis, using Compute Engine.
• ⚙️ Managed instance groups are used to scale the application and ensure availability during zonal outages.
• 🔄 Instance templates may need updates, such as operating system upgrades, A/B testing, or disk configuration changes.
• 🚢 Cymbal Superstore’s ecommerce app is containerized and deployed using GKE, requiring external connectivity configuration.
• 🌍 External HTTP(S) load balancers provide a global IP and content distribution close to the end-user.
• 📊 The transportation management app processes sensor data using Cloud Functions and a Dataflow pipeline, storing data in Bigtable.
• 📋 Queries on Bigtable data can be executed using BigQuery SQL, with the setup involving external table definitions in JSON.

Q & A

What is the role of an Associate Cloud Engineer after the deployment and implementation of cloud solutions?

The role shifts to maintaining successful operations, which includes managing resources used in cloud solutions and using Google Cloud Observability for monitoring and logging.

What are some key resources an Associate Cloud Engineer should be familiar with on Google Cloud?

They should be familiar with compute, storage, and networking resources such as Compute Engine, Managed Instance Groups, GKE Pods, Cloud Functions, and Dataflow pipelines.

Why might an instance template in a Managed Instance Group need to be updated?

The instance template might need updating to change the operating system, conduct A/B or canary testing, or change the disk type or attached disks.

What needs to be done after updating an instance template in a Managed Instance Group?

After updating the instance template, the changes must be propagated to all VM instances in the group.

What is the role of an external HTTP(S) load balancer in Cymbal Superstore’s ecommerce app?

An external HTTP(S) load balancer provides a single global IP, delivers content close to end users, and forwards content to globally available backends.

What technology does Cymbal Superstore’s transportation management app use to monitor incoming sensor data?

The transportation management app uses Cloud Functions to monitor incoming sensor data and a Dataflow pipeline to write data to Bigtable.

How does an Associate Cloud Engineer support a data engineer in the context of Cymbal Superstore’s transportation management app?

The Associate Cloud Engineer provides information about the sources and sinks required by the Dataflow pipeline to the data engineer responsible for developing it.

What steps are involved in querying Bigtable data using BigQuery SQL?

First, a permanent external table is defined using the Google Cloud Console or the bq command-line tool by creating a table definition file in JSON. Then, a reference to the external table is created with a 'bq mk' command, and finally, the query is submitted using BigQuery SQL.

What information is included in the table definition file for querying Bigtable data with BigQuery?

The table definition file includes the URI for the Bigtable, along with details about the column families and columns defined in the table.

What is the purpose of a sink in the context of Cymbal Superstore’s Dataflow pipeline?

A sink in the Dataflow pipeline writes processed data, such as incoming sensor data, to storage solutions like Bigtable for further analysis or querying.